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EXECUTIVE SUMMARY 



The European Higher Education Area (ehea) would not be what it aimed to become 
without a strong emphasis on quality and quality assurance, as the latter can provide 
the basis for mutual trust — the prerequisite without which recognition, mobility and 
institutional cooperation could not function on a larger scale. The Standards and 
Guidelines for Quality Assurance in the European Higher Education Area (esg) have 
been developed to cater for maintaining national characteristics and as a response to 
the essential need to create a mutual understanding of how to safeguard existing qual- 
ity and enhance it further. The ESG have aimed to shape the construction of the ehea 
and ensure the trustworthiness of higher education to European society. 


The results of the analysis presented in this report underline the essential need for, 
and the success of, the ESG. At the same time, the findings and recommendations also 
provide concrete suggestions for further improvement of the ESG, so that they might 
best contribute to the desired European Area of Quality Higher Education. 

The esu consultation report of the map-esg project builds on the experiences of na- 
tional and local student organisations as well as individual student representatives 
and student reviewers who have been involved in quality assurance at institutional, 
national and European level. The consultation process was conducted through struc- 
tured questionnaires, individual interviews and a focus group. Data was collected 
from 29 national student unions from 27 countries, 27 local student unions from 7 
countries, 11 individual students who took part in the external reviews of qa Agencies 
and 21 focus group participants from 15 countries, who were also interviewed individ- 
ually. The data analysis was complemented with text analysis of the ESG and relevant 
literature where appropriate. 


I 


Despite the variety of different data sources, several themes emerged as those which 
the students identified as essential to a discussion on the ESG. These themes are: the 
purpose and the scope of the ESG, clarity and usability of the ESG, impact and imple- 
mentation of the ESG and stakeholder involvement in quality assurance as a particu- 
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lar, overarching theme. A summary of key findings for each of the identified themes, 
followed by a summary of recommendations for further improvement of the ESG, is 
provided below. 

PURPOSE AND SCOPE OF THE ESG 

Key findings 

O In the view of students, the main purpose of the ESG is to provide a tool in form- 
ing a European Higher Education Area and in ensuring the provision of incen- 
tives for changes that would lead to the achievement of the goals set forth in the 
Bologna Process. 

O The ESG are regarded as highly relevant to the establishment of a sound qa sys- 
tem, both nationally and at institutional level. 

O Students attribute high relevance to the ESG to provide criteria or a basis for 
them to be used for reviews of programmes, institutions and qaas, as well as 
making external quality assurance more transparent and simpler to under- 
stand. 

O Students see it as essential to widen the scope of the ESG in order to foster the 
development of the ehea in real terms and build trust between higher educa- 
tion systems and higher education institutions alike. 

O Students see a necessity to widen the scope of the ESG in two ways. Firstly, since 
specific instruments outlined in the Bologna Process have a direct relevance 
for the quality of higher education, the standards and guidelines should be 
amended to include clearer linkage and reference to the specific principles of, 
and specific commitments undertaken within, the Bologna Process. 

O Secondly, students were of the opinion that further elements ought to be ad- 
dressed by the ESG, namely institutional management; research; lifelong learn- 
ing and transnational education. 

Key recommendation 

It is essential that the revision of the ESG is not limited only to minor amendments of 
the existing standards and guidelines, but involves a broader discussion on the pur- 
poses and the scope of the ESG. This discussion on the purposes and the scope of the 
ESG may be more demanding and challenging than a mere improvement of the current 
text of the ESG. It is, however, essential. The standards and guidelines form a holistic 


2 


ESU CONSULTATION REPORT OF THE MAP-ESG PROJECT 


document which is only meaningful in the context of its overall purpose and its de 
fined scope. 

CLARITY AND USABILITY OF THE ESG 

Key Findings 

O All the national student representatives have indicated that they are aware of 
the ESG. More surprising, perhaps, is that at the local level only a few student 
representatives have indicated that they have not heard of the ESG. The majority 
of them indicate that they have some knowledge of the ESG, even though not 
always in detail. 

O One of the identified obstacles to easy understanding and clarity of the ESG is 
the language, phrasing and, in particular, the meaning of the specific quality 
assurance terminology used in the ESG. 

O Students who have taken part in external reviews assert high familiarity and 
understanding of the ESG to the external review panel members. However, they 
highlight that the ESG also leave ample room for ambiguity. About half of them 
indicated that there was little or no shared understanding between panel mem- 
bers in the external reviews of what the ESG mean, how they should be inter- 
preted and how they should be used for reaching conclusions in reviews. 

O As reasons for the ambiguities identified in the ESG, both the students in the 
focus groups and a careful text analysis of the ESG pointed clearly to the in- 
consistency of approach to individual standards and guidelines throughout the 
ESG. The inconsistency of the ESG is most evident in: 

O significant differences in the precision of formulation of different standards and 
their accompanying guidelines. Whilst some are deemed precise, offering little 
room for interpretation, others are vague and use terms that are much more 
open to interpretation; 

O the relationship between the standards and guidelines — there are stand- 
ards which may be vague or which are not much elaborated, whilst their ac- 
companying guidelines are very precise and bring in important, additional 
elements for implementation and guidance. 

O Students noted that there is no clarity as to the intended differences between 
standards and guidelines in their practical application. In other words, if 
guidelines bring in a number of additional, important elements that are not 
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addressed in the actual standards, to what extent should they be followed, or 
indeed checked in the external reviews? 

O Furthermore, practice gives an indication that standards are assigned different 
levels of priority, and are certainly prioritised over guidelines; it seems to be a 
matter of personal interpretation as to which standards are considered core and 
which less important. This endangers consistency and an equal treatment of the 
different standards in the quality assurance process. 

O Although the ESG, via standards 2.1 and 3.1, were clearly designed to link inter- 
nal and external quality assurance, it appears that this link in practice is mal- 
functioning. 

O Finally, it can be said that the ESG as a document lack a coherent approach, re- 
sulting in inconsistencies in their application, which needs to be remedied. 

Key Recommendations 

In order for the ESG to achieve their purposes, it is essential not only to create aware- 
ness and knowledge, but also for the ESG to be easily understood by higher education 
stakeholders active at the European and, crucially, also at national and institutional 
levels. 

The clarity of language and terminology may be enhanced through elaborating further 
the differences between national contexts and approaches to quality assurance in the 
introductory text to the ESG, where more recent information and analysis of the na- 
tional qa systems could be taken into account. 

The issue of ambiguity and achieving more easily a shared understanding of the ESG 
maybe more complex to address. However, it is essential to address it, thus enabling 
consistent use of the ESG in external reviews of both higher education institutions and 
quality assurance agencies. Revising the ESG in a manner that will leave less room for 
interpretation might be a solution to this, although this might imply less room for flex- 
ibility and enforce more rigidity. 

Although not a direct solution to the issue of clarity of the text of the ESG itself, fur- 
ther suggestions for improvement include a glossary of terms in the context of the 
purposes of the ESG and a guide to the use of the ESG for external reviews and/or an 
explanatory report similar to that of the Lisbon Recognition Convention. 

To address the inconsistencies in the manner in which the standards and guidelines 
are written, a revision of the standards and guidelines as such is needed. The revision 
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of the actual individual standards and guidelines should, at the very least, address the 
inconsistencies in the level of precision in which different standards and accompany- 
ing guidelines are formulated. 

Moreover, the relationship between the standards and guidelines should be discussed, 
particularly bearing in mind that some guidelines bring in additional elements to the 
standards. Thus, the role of the guidelines, particularly in the context of institutional 
implementation and external reviews, should be clearly stated. 

IMPACT AND IMPLEMENTATION OF THE ESG 

Key Findings 

O The ESG have had considerable impact on the development of the national QA 
systems since their adoption, either directly or as a guiding document. The use 
that is sometimes made of the ESG in the development and changes of the qual- 
ity assurance systems does, however, not imply that the standards and guide- 
lines have been implemented consistently. 

O The analysis of the data collected paints a very diverse picture of the ways and 
extent to which standards and guidelines are actually being implemented with- 
in national higher education systems and higher education institutions. 

O The analysis of the information received from the national unions of students 
suggests that all quality assurance systems have similar standards to those de- 
fined in the ESG incorporated, whilst lacking a number of important elements 
of the standards or some standards entirely.. 

O A closer look at the institutional level shows that in all countries in our sample, 
diverse as they are, there are heis with a system of internal qa in line with the 
ESG, but more heis which do not have this. The responses from local student 
representatives suggest that the implementation of a qa system that follows the 
principles of the ESG relies more on heis than on national governments. This 
may mean that, where in one institution certain standards are almost complete- 
ly translated into the internal QA system, another hei in the next town might 
have a system in which a number of standards and guidelines are lacking. 

O Certain preconditions for quality of higher education have been introduced as 
standards into the ESG (such as qa of teaching staff (Standard 1.4); learning re- 
sources and student support (Standard 1.5), information systems (Standard 1.6) 
or public information (Standard 1.7)) while others are not. The fact that those 
standards containing preconditions for quality are formulated alongside stand- 
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ards that are more directly related to policies, procedures and mechanisms of 
QA suggests that these preconditions are essential to take into account for heis 
in assuring and enhancing quality of education. According to the ESG, good 
quality education is not just dependent on proper evaluation and improvement 
mechanisms, but also on a (minimum) number of preconditions that need to be 
satisfied to enable the provision of good quality teaching and learning. It is how- 
ever questionable why only some of such crucial preconditions are mentioned 
in the standards and guidelines while others — e.g. stakeholder involvement 
which is only mentioned in the explanatory text or the link between teaching 
and research, which is only referred to in the guidelines, are not. 

O Although most local student representatives believe that the information their 
heis provide to the wider public is impartial and objective, they also report that 
this information is more often primarily intended as a marketing tool than as a 
way of objectively informing (future) students and the wider public. 

O With regards to student involvement, the textual analysis of the ESG suggests 
that regardless of referring to it as student involvement or student participation, 
or even student feedback, student engagement (yet another term often used) 
in qa can mean different things to different people. Data obtained through the 
local student representative survey seems to support the impression that stu- 
dent participation is only relevant, and encouraged, in areas of qa where it is 
explicitly mentioned in the ESG. 

Key Recommendation 

Student involvement should be defined more explicitly and consistently, both as a 
principle and within different standards, to fully integrate the concept of students as 
full partners and active actors in university and higher education decision-making 
processes, particularly those in qa. The Ministers’ summit in Bergen in 2005 provid- 
ed initial guidance for defining student participation, whilst the Stocktaking report 
already presented student involvement in qa. This was achieved by specifying that 
student participation in the governance of national bodies of QA; within teams for 
external review; consultation or involvement during external reviews; and involve- 
ment in internal evaluations (Bologna Process Stocktaking, 2005, p. 17) should be 
implemented. 

Furthermore, for students it is also of utmost importance that their role would be ac- 
knowledged in the decision-making processes within higher education institutions 
and national education systems as full partners in the development of quality higher 
education. This means that students should be involved in the governance structures 
in higher education institutions and should be consulted at the national level. In the 
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context of this report and in implementing the ESG, students should be involved in the 
setting up of both national and institutional quality assurance systems and take an ac- 
tive part in the quality assurance bodies created either nationally or institutionally. 

STAKEHOLDER INVOLVEMENT 

The ESG are a showcase for the unique method of cooperation that is a crucial factor 
for the success of the reforms in European higher education: the involvement of stake- 
holders. The ESG are the result of broad cooperation; developed by higher education 
institutions, students and quality assurance agencies alike; adopted by Ministers re- 
sponsible for higher education. This has ensured their acceptance in the higher educa- 
tion community and beyond. 

Therefore, the discussion on the future of the ESG should be based on the same princi- 
ple — the input provided through the consultations of each of the E4 partners provides 
an in-depth insight into the ways in which different stakeholders perceive the purpose 
and scope of the ESG and how they evaluate the ESG against these purposes, as well as 
recommendations on how to further improve the ESG to best cater for them. As in any 
inclusive process, the views of different stakeholders sometimes differ and sometimes 
converge, and it is in the differing views that the future of European cooperation in 
quality assurance is hidden. Through a discussion on the results of each of the consul- 
tation reports, and the overall E4 consultation report, a clear mandate to the E4 should 
be elaborated to allow for the further improvement of the ESG and the European Area 
of Quality Higher Education. 


FOREWORD 

Quality as a concept is indispensable for all who are concerned with an educational 
system, particularly with higher education. Not only is quality a distinguishing factor 
for higher education institutions and those individuals who take a direct interest in it; 
society at large relies on an understanding of quality as the core element that nurtures 
scientific, technological, social, economic and cultural advancement. It is specially 
what students are deeply concerned about, as higher education that is of low quality 
is meaningless. 

Quality assurance therefore has been one of the main topics in the European discus- 
sions on developing and reforming the tertiary educational system. The European 
Higher Education Area (ehea) that is emerging would not become what it aims to 
be without strong emphasis on quality and quality assurance; the latter can provide 
the basis for mutual trust — the prerequisite without which recognition, mobility and 
institutional cooperation could not function on a large scale. There are, of course, sev- 
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eral approaches to quality assurance; thus the main goal within the Bologna Process 
has been to focus on cooperation in quality assurance, rather than to design one single 
system for the whole of Europe. National systems still allow for better consideration of 
the respective context, while naturally being an outcome of the traditional evolution 
of a given system. 

The European Standards and Guidelines (esg) have therefore been the response to 
both catering for the maintenance of national characteristics and the essential need 
to create a mutual understanding of how to safeguard existing quality and its further 
enhancement. The ESG have aimed to shape the construction of ehea, and ensure 
the trustworthiness of higher education to the European society, thereby also provid- 
ing an alternative to systems that are highly heterogeneous in terms of quality. The 
ESG are also a showcase for the unique way of cooperation that is a crucial factor for 
the success of the reforms in European higher education: the strong involvement of 
stakeholders. The ESG are the outcome of a widely based cooperation; developed by 
higher education institutions, students and quality assurance agencies alike; adopted 
by Ministers responsible for higher education. This has ensured their acceptance in 
the higher education community and beyond. 

One major characteristic of quality assurance is the approach of continual enhance- 
ment — a light in which also the ESG should be viewed in. Thus the ESG that were 
adopted in 2005 had to be analysed in terms of their practical application in order to 
ascertain where there is further room for improvement; how they would best cater for 
the purposes for which they were intended, and also to see whether they have been 
successfully used for other purposes and whether further enhancement there would 
be meaningful.. 

Essential in this, again, is the involvement of students, as those being directly con- 
cerned and affected by the quality of education. Based on broad consultation, this 
report analyses the views of students from all over Europe, including representatives 
from the grassroots level in individual universities, the national unions of students, 
and student peer reviewers and experts in quality assurance alike. 

The results of this analysis are presented in this report and underline the essential need 
for, and the success of, the esg. At the same time, the findings and recommendations 
also offer definite suggestions for further improvement of the ESG, so that they might 
best cater for the desired European Area of Quality Higher Education. 
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1.2 INTRODUCTION 


The esu consultation report of the map-esg project builds on the experiences of na- 
tional and local student organisations as well as individual student representatives 
and student reviewers who have been involved with quality assurance (qa) on the 
institutional^ national and European level. The consultation process was conducted 
through structured questionnaires, individual interviews and a focus group. Data was 
collected from 29 national student unions from 27 countries, 27 local student unions 
from 7 countries, 11 individual students who took part in the external reviews of qa 
Agencies and 21 focus group participants from 15 countries, who were also individually 
interviewed. The data analysis was complemented with text-analysis of the ESG and 
relevant literature where appropriate. 

The initial analysis of the gathered data has shown that the ESG have had an impact 
on national and institutional quality assurance systems. Furthermore, it showed that 
a number of consistent issues are raised in connection to the ESG, although the data 
were gathered from three different sources and were complemented by literature re- 
view and text analysis. Therefore, to ensure an evaluative approach to the analysis, as 
well as to provide an in-depth analysis of the identified issues that enables problema- 
tising rather than simplification of issues, a thematic approach to the esu report was 
chosen. In other words, the analysis of the data and emerging conclusions are struc- 
tured around the identified issues. 

The report itself starts with a description of the methodology used. It is then divided 
into three main sections: the first focussing on the purpose and scope of the ESG; the 
second on usability and clarity of the ESG and the third on implementation and impact 
of the ESG. Within these sections, the identified issues are addressed. These comprise 
the following areas: awareness and knowledge; understandability; relevance and pur- 
pose; impact; scope; relationship between different parts of the ESG and the relation- 
ship between standards and guidelines and stakeholder involvement. It then provides 
a summary of the key findings with the last part consisting of the conclusions and 
recommendations stemming from the analysis. 


1.3 OUTLINE OF THE METHODOLOGY 

The aim of the esu consultations on the ESG was to bring together the full range of 
student experiences with quality assurance processes, with particular attention being 
paid to the impact and use of the ESG in different contexts — national, institutional 
and within external reviews. The aim of the consultation process was to evaluate the 
ESG in terms of their understandability, clarity and completeness, as well as to map 
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their implementation and impact on the institutional, national and quality assurance 
agencies level. 


1.4 THE RESEARCH TEAM 

The research team consisted of Bastian Baumann, a former member of the esu Bolo- 
gna Process Committee (2002-05), former Secretary General of the Magna Charta 
Observatory and currently independent higher education consultant; Nik Heerens, 
former member of the esu Executive Committee (2003) and Bologna Process Com- 
mittee (2003-05), former manager of higher education projects of spark and Head 
of Student Participation in Quality Scotland (sparqs), currently PhD researcher at 
the University of Exeter and Combined Universities in Cornwall (cue); Vanja Ivose- 
vic, former Chairperson of esu (2005) and esu Executive Committee member (2003, 
2004), former Deputy Director of the Centre of Education Policy (cep, Belgrade) and 
currently project manager at the Institute for Development of Education (ide, Za- 
greb). 

The research team was coordinated by Andrea Blattler, former member of the esu Ex- 
ecutive Committee (2009/10) and the esu Academic Affairs Committee (2010/11) and 
current student of political science and philosophy and undergraduate research and 
teaching assistant at the University of Lucerne and advised by Allan Pall, former ESU 
Vice-Chairperson (2009/11) and Chairperson (2011). 

The research team was responsible for conducting the esu consultation; for the de- 
velopment of the consultation methodology and instruments and for analysing and 
interpreting the data and writing the ESU consultation report. This team therefore 
bears full responsibility for this report. 

THE CONSULTATION PROCESS 

The esu consultation process aimed at collecting data from three different groups of 
students, to encompass the range of experiences with quality assurance processes and, 
in particular, on the impact the ESG have on different contexts in which they are used. 
Therefore, the research builds on the experiences of national and local student organi- 
sations, student reviewers who have been involved in the external quality assurance 
of quality assurance agencies, as well as individual student representatives who have 
gained experience in quality assurance on the institutional, national or European lev- 
el and qualify as members of the esu quality assurance student experts’ pool. 

The empirical part of esu’s consultation for the map-esg project was undertaken be- 
tween February and July 2011 and consisted of a combination of different research 
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techniques for data collection; namely sizeable structured questionnaires, semi- 
structured interviews, focus groups and a review of literature. Thus a combination of 
qualitative and quantitative data was gathered. 

Members of the ESU quality assurance student experts pool 

In February 2011, esu invited individual student members of the esu quality assur- 
ance student experts’ pool to take part in a map-esg consultation workshop in Mainz, 
Germany. Participants were selected following a call to all the members of the esu 
quality assurance student experts pool, with the aim of reaching a diversity of experi- 
ences as well as selecting a regional and gender balance of the group. Participants were 
asked to describe their experience in quality assurance and what they considered to be 
their main contribution to the workshop beforehand) as well as regional and gender 
balance of the group. 

The goals of the workshop were primarily to gather the students’ experience regard- 
ing the understandability, clarity and completeness as well as the implementation and 
impact of the ESG on the different levels of quality assurance. The secondary aim was 
to refine the consultation methodology on the basis of the feedback of student quality 
assurance experts. 

The workshop gathered 21 individual students from 15 different countries who took 
part in plenary discussions and focus groups. The research team also interviewed the 
21 students individually. 

National unions of students 

Between May and July 2011, 45 national unions of students from 38 European coun- 
tries (all the members of esu) were consulted, using a structured online question- 
naire using open-source survey software (Limeservice). This questionnaire aimed to 
collect (1) the views of the national unions of students on the ESG, (2) information on 
the way in which the ESG have (or have not) influenced the national quality assurance 
systems, (3) the views of the national unions of students on clarity, comprehension, 
relevance and implementation of each of the standards and accompanying guidelines 
and (4) the views of the national unions of students on whether and, if so, how the ESG 
should be revised. 

In total, 29 national unions of students from 27 countries answered the questionnaire, 
which amounts to a response rate of 64.5%. The responses were submitted anony- 
mously. 
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Local student representatives 


With the aim of getting a perspective of higher education institutional quality assur- 
ance all over the ehea, the research team decided to consult a sample of local student 
representatives, esu is an umbrella organisation of the national unions of students in 
Europe — the members of the national unions are mostly local student unions based 
in higher education institutions of the respective countries. Taking into account the 
limited time of 6 months available for the entire study, the research team selected a 
sample of countries from which local student unions were included in the consulta- 
tion process. 

Seven countries were selected, covering a wide diversity of higher education systems 
and a diversity of quality assurance systems while ensuring a regional balance. There- 
fore, the sample includes binary and unified, as well as small and large higher educa- 
tion systems and with quality assurance systems designed before and after the ESG 
were adopted, in addition to those based on institutional reviews and those including 
programme reviews. The selected countries were: The Netherlands, Norway, Serbia, 
Slovenia, Spain, Switzerland and the United Kingdom (therein: Scotland and Eng- 
land). 

Within each of those seven countries a stratified random sample of a minimum 20 % of 
all heis was selected, taking into account specificities of particular higher education 
systems. Namely, in binary systems, both types of heis were included; heis from all 
different linguistic regions in multilingual higher education systems and from all re- 
gions in countries with decentralised government were included. As agreed with the 
other E4 partners, some higher education institutions which belonged to the consulta- 
tion sample of the European University Association were added to the selected sam- 
ples in order to enhance comparability between the different consultation outcomes. 

Local student unions in the selected higher education institutions were contacted, re- 
gardless of their association with the national unions of students, which are members 
of esu, although the national unions assisted in providing contacts and support letters 
to increase the response rates. In order to increase accessibility, the structured ques- 
tionnaire was available in six languages (English, French, German, Serbian, Slovenian, 
and Spanish). 

In total 27 responses were received; these cover between 10 and 50 % of the higher edu- 
cation institutions in each higher education system, apart from England and Norway 
where the response rate was lower, partly due to the overall number of higher educa- 
tion institutions in the higher education system. 
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Individual student reviewers of quality assurance agencies 


To receive first-hand experience of the external reviews of quality assurance agencies, 
28 students, who have all been members of panels reviewing quality assurance agen- 
cies, were consulted through a structured questionnaire in June and July 2011. The 
questionnaire addressed the individual experience of the students in the external re- 
view team; relevance and adequacy of the ESG as a whole; and individual standards and 
guidelines in the reviews of qa agencies; and inquired about the views of the students 
on the necessity and the way in which the ESG could be revised and/or improved. 

11 of those individual student reviewers answered the questionnaire, which consti- 
tutes a response rate of 39.3 %. 

The information obtained through questionnaires, interviews and focus groups was 
complemented with text-analysis of the ESG and review of relevant documents on qa 
and the ESG. 
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2 SECTION 1: 

PURPOSE AND SCOPE OF THE ESG 


2.1 PURPOSE AND RELEVANCE 

The ESG have been developed with a range of purposes and objectives in mind. There 
has been no clear definition as to what would constitute the difference between pur- 
poses and objectives. Therefore, for the purpose of this study, no distinction has been 
made between those terms and the authors refer only to purposes of the ESG. In this 
sense, the purposes stated in the ESG themselves are: 

O »to improve the education available to students in higher education institu- 
tions in the ehea; 

O to assist higher education institutions in managing and enhancing their quality 
and, thereby, to help to justify their institutional autonomy; 

O to form a background for quality assurance agencies in their work; 

O to make external quality assurance more transparent and simpler to under- 
stand for everybody involved; 

O to encourage the development of higher education institutions which foster vi- 
brant intellectual and educational achievement; 

O to provide a source of assistance and guidance to higher education institutions 
and other relevant agencies in developing their own culture of quality assur- 
ance; 

O to inform and raise the expectations of higher education institutions, students, 
employers and other stakeholders about the processes and outcomes of higher 
education; 
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O to contribute to a common frame of reference for the provision of higher educa- 
tion and the assurance of quality within the ehea« (esg, p. 14). 1 

However, further purposes might be relevant which, although possibly not intended 
initially, have become de facto purposes of the ESG through the use of the ESG, such as 
facilitating the establishment of a sound national system of qa, or of a sound institu- 
tional QA system of heis. Another purpose that was not specifically mentioned is that 
the ESG might provide criteria for checking compliance in reviews of programmes, in- 
stitutions and qaas. 

Additionally, some purposes might have been initially intended, but — for different 
reasons — were not stated in the ESG, such as the encouragement of a free market in qa 
(c/. European Commission, 2004) or the protection of students from accreditation or 
degree mills, (c/. Williams, 2008). 

The findings of the surveys clearly indicate that the ESG fulfil a range of purposes. Re- 
spondents both at the national and the local level listed several purposes that they 
found relevant and there is a large overlap as regards the relevance of the purposes 
by both respondents from the local level and the national level. Discrepancies mainly 
relate to the different focus of work of student representatives at local or at national 
level. Therefore, the relevance of the ESG for forming a background for the work of 
qaas is deemed higher for national level representatives than for local ones, whereas 
the encouragement of hei development and the improvement of quality for students 
is deemed more relevant at local level than by the respondents from the national level. 
Local union representatives apparently see the quality enhancement function of the 
ESG having an effect on the quality of students’ education, whereas national level rep- 
resentatives focus rather on the enhancement of the qa system as such, thereby indi- 
rectly affecting the quality of education available to students. 

The ESG are generally considered highly relevant for the establishment of a sound 
QA system. Local respondents put a greater emphasis on internal qa systems of heis 
whereas national respondents deem the ESG more relevant for the national QA system; 
this was to be expected for the above-mentioned difference in focus of work of both 
respondents. 

Local and national respondents also attribute high relevance to the ESG, providing 
criteria for reviews of programmes, institutions and qaas, to making external quality 


1 All references to the ESG in this report are related to the 3rd edition of the Standards and 
Guidelines for Quality Assurance in the European Higher Education Area, http://www.enqa. 
eu/files/ESG_3edition %20(2).pdj 
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assurance more transparent and simpler to understand and to contributing to a com- 
mon frame of reference for the provision of he and QA in the ehea. 

The national level respondents view the promotion of a quality culture through the 
ESG more relevant than their local respondents. This might be due to greater aware- 
ness of the discourse on, and higher familiarity with, the term of >quality culture< at 
the national level. It might also be that local respondents do not see a greater benefit 
stemming from the ESG, in terms of the standard of quality culture being developed at 
the level of individual institutions. This would then however raise the question of how 
much the ESG in reality contribute to the development of an institutional quality cul- 
ture since, arguably, local student union representatives should best be able to judge.. 

The encouragement of a free market in QA in the ehea is considered to be of the lowest 
importance both for local and national student representatives. This finding did not 
surprise, as students are generally fairly reserved regarding a free market approach in 
higher education. However, the protection of students from degree and/or accredita- 
tion mills is also seen to be relatively less relevant in their opinion. An explanation for 
this might be that such protection might be considered a purpose of the eqar, and 
some other organisations, rather than of the ESG themselves. 

When asked about the main purpose of the ESG, the respondents’ answers tended to 
the common frame of reference for the provision of higher education and the assurance 
of quality that would be created in the ehea and the provision of incentives for change 
that is inherent in this. Many believe that, without the ESG, certain attributes would 
only be incorporated significantly later, or not at all, into the national systems, such as 
the notion of quality culture, student participation in QA or generally new approaches 
to QA in the respective systems. As one of the interviewed students explains: 

»Previously students were not members of panels and there was no student input to 
the self-assessment and the external evaluation reports. External evaluation reports 
were basically a copy of the self-assessment reports written by the higher education 
institutions themselves, eqar and enqa membership influenced the higher educa- 
tion institutions and the government to include the students. « 

This does not, however, guarantee that through the ESG they are incorporated (see 
part on stakeholder involvement in Section 3), but without the guidance and soft pres- 
sure coming from the ESG, the chance of inclusion would be even less. Thus, in terms 
of functions that the ESG fulfil at the national and local level, the students view the 
mechanisms of soft pressure and guidance as important, in addition to the use of the 
ESG as criteria for reviews. 
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Also, the common frame for qa in the ehea was stated as an important purpose for 
different reasons. The ESG should ensure comparability and compatibility between 
systems, whilst allowing for adaptations for the national context, which implies that 
the ESG should not be too rigid. But they are also seen as a mechanism to prevent ad 
hoc changes to the national system that might occur due to political changes in the 
country which might endanger stability. Hence, the ESG should provide a constant in 
national he systems. 

From the survey and focus groups it became obvious that students at both local and 
national level view the initially-stated purposes of the ESG but also other, de facto 
purposes, as important.. For example, the students who took part in the focus group 
commented that the ESG could contribute to stakeholders using a similar language 
and understanding regarding quality assurance; and that the ESG might promote dis- 
cussion on quality and good practice within higher education institutions, as well as 
contributing to the development of trust between both higher education systems and 
higher education institutions. This, in turn, poses the question of whether the ESG 
are suitable to fulfil these additional purposes, given that they might not have been 
designed for them. This seems particularly relevant regarding the use of the ESG as a 
reference checklist in assessing compliance, (which, it is often said, the ESG are not in- 
tended to be). Interestingly, the esgs provision of criteria was seen by national unions 
of students as the best outcome which, as noted above, was also thought to be highly 
relevant.. In addition,, the fact that that the ESG facilitate the establishment of sound 
internal qa systems has been seen by most as fulfilled, whereas fewer thought this to 
be the case with the facilitation of the establishment of a sound national qa system. 
Apart from the provision of criteria, the purposes that have been seen as most fulfilled 
coincide only partially with those thought most relevant. The contribution to a com- 
mon frame for the ehea, and that the ESG form a background for the work of qaas, has 
been regarded as fulfilled to a large extent. 

However, the ESG live up neither to the expectations regarding greater transparency 
and easier understanding of external qa, nor to the idea that they improve education 
available to students. This is to be seen as one of the major shortcomings of the ESG, as 
these purposes themselves have been deemed very relevant and are often referred to 
in current debates on qa. The lack of fulfilment of these purposes gives rise to ques- 
tions on how greater transparency and a more direct effect on the improvement of 
education available to students could be achieved. 

One of the highly relevant, and at the same time apparently fulfilled, purposes of the 
ESG is to provide criteria for reviews of quality assurance agencies and guidance for 
institutional or programme reviews. This however stands in contrast with the impres- 
sion that the ESG were not meant to serve as a compliance tool initially. It is question- 
able whether this purpose of the ESG is in conflict with other purposes of the ESG. It 
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seems adequate to encourage a discussion about bridging this apparent discrepancy. 
Furthermore, some comments have suggested that the ESG should be further devel- 
oped to serve as a tool for establishing national and/or institutional systems of QA, or 
to provide a guarantee for certain features of the system, whilst other some respond- 
ents have argued that the ESG must not become too regulatory. This disparity poses 
a great challenge to the ESG. The ESG should allow for both sufficient flexibility and 
adaptability to cater for diverse national and institutional contexts. At the same time, 
they should be sufficiently well-established to ensure comparability and compatibility. 
Furthermore, they should also have a supporting character for designing and revising 
QA systems. The latter may not necessarily be an issue of the ESG themselves, as they 
might not be appropriate for this, but a discussion about the guiding factor of the ESG 
would seem advisable. 

Regardless of their differences, there does not appear to be a contradiction between 
the purposes of the ESG. It is perceived by students to be one of the major success fac- 
tors of the ESG that they are able to cater for different purposes. Even if the ESG are 
seen to fulfil their individual purposes in different ways, they do not constitute an 
obstacle to achieving any of the purposes. 

The general relevance of the ESG is undoubted. All respondents view the relevance of 
the ESG at the European level as high and a very large number also believe that the ESG 
are relevant at the national level, although fewer than at the European level. This also 
corresponds to the previous views of the national unions expressed in the Bologna 
With Student Eyes publication of 2009 (esu, 2009), thus indicating that the ESG have 
had on going influence. 

Regarding the perceived relevance of individual standards and guidelines, there have 
been few differences. A great majority of the respondents judged all standards and 
their accompanying guidelines as equally important. However, the relevance of in- 
dividual standards has only been surveyed in connection to the review of qaas and 
the functioning of QA within heis and not in any other context. There is, nevertheless, 
a difference between the importance of the standards given in review processes, as 
some standards are deemed more important than others. This inconsistency is being 
addressed in Section 2. 

The ESG are certainly a very visible feature of European cooperation in quality assur- 
ance. Such European cooperation in quality assurance was almost unanimously de- 
clared a necessity, also by some respondents from the local level who had not heard 
of the ESG before. Some other local student representatives who have not heard of the 
ESG before were the only ones to state that European cooperation in QA is not neces- 
sary. Only some of the respondents from the local level who were not previously aware 
of the ESG, believed that European cooperation in QA is not necessary. 
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2.2 SCOPE OF THE ESG 


As stated in the ESG »the standards and guidelines relate only to the three cycles of 
higher education described in the Bologna Declaration and are not intended to cover 
the area of research or general institutional management« (esg, p. 12). There is no fur- 
ther information on how the scope of the ESG was determined by the E4 or argumenta- 
tion for the scope chosen, although the mandate given to E4 by European Ministers 
did not specify the scope of these standards and guidelines; it was said only that the 
E4 should »develop an agreed set of standards, procedures and guidelines on quality 
assurance, to explore ways of ensuring an adequate peer review system for quality as- 
surance and/or accreditation agencies or bodies« (Berlin Communique, 2003, p. 3). 

Furthermore, the ESG were developed between the Ministerial summit in 2003, at 
which the E4 was mandated to prepare the ESG, and the Ministerial summit in 2005, 
at which the ESG were adopted by the Ministers. Therefore, the ESG could have only 
taken account of the Bologna Process developments that were relevant at that time. 
However, it is important to note that the ESG, although developed within the Bolo- 
gna Process framework, do not specify specific standards on agreed Bologna Proc- 
ess instruments, such as the implementation of the ects and the issuing of Diploma 
Supplements, which were already agreed in 2003. It might be argued that the specific 
standards on these issues would go against the preferred »generic principle to the spe- 
cific requirement of the ESG (esg, p. 11). But that has not been followed consistently 
throughout the ESG; other areas related to quality of higher education — for example, 
the information systems which heis use for their quality management and the infor- 
mation they should provide to the wider public — have been described in considerable 
detail in, respectively Standard 1.6 Information systems and 1.7 Public information 
and their accompanying guidelines. 

Furthermore, the general principles of the Bologna Process already agreed in 2003, 
such as the importance of mobility or employability of students, could also feature 
more prominently in the ESG, as also noted in the European Commissions report to 
the Council on progress in quality assurance in higher education (European Commis- 
sion, 2009). A reflection of these principles would obviously not mean a translation 
of these principles into standards par for par but rather the acknowledgement that in 
order to reach the goals of the Bologna Process, certain conditions and tools need to 
be implemented interlinked to and within systems of quality assurance. 

The key issue is, thus, whether the ESG should be encouraged to widen its scope, which 
move the national unions of students appear to support. The individual students who 
took part in the focus group concur with this view. 
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As reported by the national unions of students, the ESG have had an impact on the 
national systems of quality assurance, and are viewed as an important instrument for 
building a common European Higher Education Area. Similarly, the individual stu- 
dents who participated in the focus group and the external reviewer’s survey see that 
the ESG are utilised in the external reviews of both higher education institutions and 
quality assurance agencies. The students also view the ESG as a relevant instrument, 
which influences the development of institutional quality culture and guidance for 
development of internal procedures of quality assurance agencies. Consequently, in 
the view of the students, widening the scope of the ESG properly to reflect the inter- 
national developments in higher education systems, particularly within the Bologna 
Process, is essential. 

Although some of the issues which the students believed to be relevant in widening 
the scope of the ESG are referred to in the introductory parts of the ESG, the students 
saw a need for addressing them in actual standards and guidelines through creating 
additional standards and guidelines and developing further the existing standards 
and guidelines where appropriate. One of the reasons was that the students agreed 
that the introductory parts of the ESG were often not taken into account as relevant for 
implementation neither nationally or institutionally. Secondly, having in mind that 
in students’ view without the ESG certain attributes of quality assurance, such as new 
approaches to qa, the notion of quality culture or student participation, would only 
be incorporated significantly later or not at all into the national systems, the students 
see that the particular standards and guidelines do have an impact on national and 
institutional practices (even if not consistently as described later in the section on 
Impact of the esg). 

The students see the necessity to widen the scope of the esg in two ways. On one hand, 
within the current scope of the ESG, the commitments taken within the Bologna 
Process should be made more explicit. Specific instruments outlined in the Bologna 
Process have a direct relevance for the quality of higher education and should be ac- 
knowledged as such in the ESG. These instruments, in the view of students, in particu- 
lar include, amongst other, the commitment to learning outcomes as introduced in 
the Berlin Communique (2003, p. 4) and further elaborated and reconfirmed in the 
following Bologna Communiques; qualification frameworks; ects and the Diploma 
Supplement. Furthermore, the students see the need for overarching principles of 
the Bologna Process to be more clearly specified within the ESG. These would, in the 
view of students, primarily include the commitment to access to higher education; 
the social dimension of the Bologna Process; student-centred learning; lifelong learn- 
ing and the mobility of students and staff. In terms of the ESG, this would mean that 
the standards and guidelines should be revised or improved either through amending 
the current standards and guidelines or developing new standards and guidelines that 
would address the above mentioned principles. 
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In order to address the social dimension of higher education generally and questions 
of access in particular, more prominent references should be made on the transpar- 
ency of he access procedures, support structures within higher education institutions, 
socially inclusive teaching, learning and assessment practices, etc. Furthermore, the 
overall importance of taking into account access and the social dimension of higher 
education in developing internal and external review processes should be emphasised 
in the ESG. 

In order to address student-centred learning, more prominent references should be 
made on the focus on learning outcomes in programme development, procedures 
for evaluation of teaching methodologies, procedures supporting the teaching staff 
in developing their teaching styles and methodology and the importance of these in 
the selection of teaching staff. Overall, a more prominent place should be given to 
student-centred learning in the development of internal and external review proc- 
esses as defined in the ESG. 

In order to address lifelong learning, clear references should be made to institutional 
approaches to adapting to the concept of lifelong learning, in particular through adap- 
tion of student support systems, access policies and student centred learning. Further- 
more, a clear reference should be made to recognition of prior learning procedures as a 
corner stone of a lifelong learning approach. 

In order to address mobility of students and staff more prominently, references could 
be made on the importance of recognition procedures within higher education in- 
stitutions being in line with the Lisbon Recognition Convention, as well as the insti- 
tutional support mechanisms for outgoing and incoming students and staff. Also, a 
clearer reference to mobility of students could be made where internationalisation is 
mentioned within the existing ESG. 

Moreover, the students saw a number of other issues of relevance that would widen the 
scope of the ESG as such. Students identified three main areas where they see a need 
to elaborate further the European cooperation in quality assurance through the ESG. 
These are institutional management, research and transnational education. 

Students saw the quality of institutional management as one of the key determining 
factors to the quality of higher education provided to students. This is consistent with 
one of the basic principles mentioned in the introduction to the ESG, which states that 
»there need to be efficient and effective organisational structures within which [the] 
academic programmes can be provided and supported« (esg, p. 14 ). Still, students 
did reflect on the importance of the ESG to allow for institutional autonomy and dif- 
ferences in the national contexts. Therefore, with regard to institutional management, 
students saw a need for institutional management to be addressed in the ESG them- 
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selves not by providing specific standards on particular governance or management 
models which would in students views hinder the principles of both institutional au- 
tonomy and adaption of ESG to national context, but through specifying standards 
on principles on which good management and governance are based on. Primarily, 
this would refer to developing standards that would support further the principles of 
transparency of decision-making and stakeholder involvement, particularly student 
participation in formal and informal decision-making. 

Particularly with the development of the European Research Area, students saw the 
need to discuss the necessity to address research as an emerging issue in the ESG. Al- 
though there were not many concrete suggestions as to howto develop the ESG to cover 
research, students saw as the very least a need to address the links between education 
and research more explicitly in the ESG due to obvious linkages and interdependency 
of teaching and learning and research in many institutions. Students more broadly 
questioned how the quality of research is measured mostly by quantitative methods 
at the moment (with a strong emphasis on number of publications and citations). Dis- 
cussion on how the actual quality of research and its connection with education could 
be assured and developed further seems highly relevant. 

Although the ESG document acknowledges that it »recognises the importance and 
implications of internationalisation for the quality assurance of higher education« 
(esg, p. 28) in 2005 it was »considered too early to include a reference to this in the 
proposed European standards for external quality assurance« (ibid). The area of in- 
ternationalisation is still of great importance to quality assurance of he and now, six 
years later, a reference to it in the ESG seems appropriate. In particular with regard 
to transnational education, students raised concerns as to whether the ESG actually 
apply to provision of higher education only within Europe or whether the European 
heis quality assured in line with the ESG are expected to respect the ESG when provid- 
ing education outside Europe. Addressing this issue may not require the development 
of new standards and guidelines, but could be reflected upon in the introduction and 
purposes of the ESG. While discussing this issue it would be beneficial to take into ac- 
count the UNESCO -OECD »Guidelines on Quality provision in cross-border higher ed- 
ucation« (unesco-OECD, 2005) and perhaps relate the two documents, which would 
also contribute to the development of greater international cooperation in quality as- 
surance and to the external cooperation of the ehea. 

Having in mind the high value the students attach to the purpose of the ESG to be the 
common frame for qa in the ehea it seems even more important to discuss the revi- 
sion of the ESG in the context of ESG purposes and scope. (See section on Purposes 
of the ESG.) Although students do agree that ESG should allow for adaptations to the 
national contexts, they also do believe that the ESG should provide for a constant in 
the national he systems as well as to ensure comparability and compatibility between 
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higher education systems. Therefore, as esg have been adopted by the Ministers in 
the framework of the Bologna process, relating more clearly the elements and instru- 
ments, as well as principles of the Bologna process which are central to the quality of 
higher education more clearly in the ESG would, in the view of students, contribute 
to faster and smoother implementation of the Bologna process nationally and would 
contribute to the Bologna goal of »achievement of greater compatibility and compa- 
rability of the systems of higher education« as stated in the Bologna Declaration (1999, 
p. 4) and reconfirmed in all the following Bologna Communiques. 
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3 SECTION 2: 

USABILITY AND CLARITY OF THE ESG 


3.1 AWARENESS, KNOWLEDGE AND COMMUNICATION 

The ESG are an outcome of the European cooperation in higher education generally 
and on quality assurance in particular. They were meant to have a direct impact on 
institutional procedures and on development or establishment of national systems. 
This is evident from the fact that they have been adopted by European Ministers in 
the context of the Bologna reforms. However, this necessitates a general awareness of 
their existence, knowledge about their intentions and of the individual standards and 
guidelines themselves. Hence, it is also a question of communication whether aware- 
ness is created and knowledge generated amongst national and institutional stake- 
holders in charge of quality assurance. 

The ESG have been translated into a number of languages 1 . These efforts have had posi- 
tive impacts on the application of the ESG, as a larger share of the target group is able to 
work with the ESG directly. However, the data analysed suggest that there is no appar- 
ent correlation between knowledge of the existence of the ESG and them having been 
translated into a working language used in the higher education and qa system. 

All national student representatives have indicated that they are aware of the ESG, with 
the majority being well aware of them. At the local level, only few have indicated that 
they have not heard of the ESG. The majority has only some knowledge of it, however, 
not in detail. 

The vast majority of the respondents also indicate that they are familiar with the qa 
procedures in place at their institutions. However, there are a number of cases where 
respondents have indicated that they know that a certain policy (in line with the esg) 
is in place, but are not sure what this policy is concretely. Hence, there seems to be a 
fairly good awareness and in parts also knowledge about qa practices at institutional 


2 Further information available at http://enqa.eu/pubs_esg.lasso 
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level that are (partly) in line with elements of the ESG. However, the lack of detailed 
knowledge of the ESG of local student representatives makes it difficult to adequately 
answer whether compliance of heis with (parts of) the ESG is directly caused by the 
existence of the ESG or because the elements described in the standards and guide- 
lines that are being implemented are seen as crucial even without the ESG. This issue 
is further addressed in Section 3. 

Student representatives at the local level have greater knowledge and awareness of 
national qa regulations than of the ESG generally. In cases where national regulations 
are compatible with the ESG, this does not automatically have a negative impact. How- 
ever, in cases where national regulations are not (fully) in line with the ESG, it remains 
difficult to foster changes at the institutional level to comply with the ESG, especially 
if an institution complies with the national regulations. Communication about the 
internal system of qa remains crucial for the institutional level. This does not mean 
that everybody needs to know the ESG literally and how the internal qa system relates 
to this, but more efforts should be made to increase awareness and knowledge about 
the implications the qa policies and practices have at institutional level. 

Students who have undertaken qaa reviews assert a high familiarity with the ESG to 
the other review panel members they have worked with. However, only about half of 
them also indicate that there is a shared understanding of what the ESG mean, how 
they should be interpreted and how they should be used for reaching conclusions in 
case the ESG are being used as a direct reference in reviews. As one of the interviewed 
students who took part in an external review of a quality assurance agency, coordi- 
nated by enqa, describes: 

»It was a very interesting experience. I had a picture of how a quality assurance agency 
works. In my country they deal with evaluations, but also information systems, admis- 
sion procedures, they have a wide range of tasks, but also personnel, a large adminis- 
tration. In this agency they had 5 employees and a task to evaluate all programmes in 
the country in 4 year cycles. Also the other members of the team came from countries 
with a very different QA system than the one we were evaluating. It was a strange ex- 
perience for everyone. 

ESG part 3 is very carefully set up to work for all agencies. We really needed to look 
into exact criteria. In the reviewed quality assurance agency, the agency, for instance, 
didn’t take any decisions. External board was taking the decisions. Since in our sys- 
tems this was not the case, and the criteria are unclear we wrote that the Agency is 
compliant with the standard if this is the interpretation and we gave our interpretation 
of the independence. If there is another interpretation, we wrote that we don’t know 
if there is compliance. « 
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This shows that even though a review team is well aware of and knowledgeable about 
the ESG, judgements and views might still significantly differ due to a different under- 
standing or interpretation. Whilst the ESG should allow for flexibility and not become 
too restrictive, it also poses problems that findings might very much depend on the 
teams or rather the individuals in teams. Hence, problems of equality and reliability 
could come up, as criteria are not interpreted and thus applied in a consistent man- 
ner. 

The ESG have not been explicitly developed with a view to be used as checklists in ex- 
ternal evaluations, but as they are de facto used for this purpose, it seems appropriate 
to try to find solutions for overcoming inconsistencies in their application. Revising 
the current ESG in a manner that will leave less room for interpretation might be a solu- 
tion to this. But this would also imply less room for flexibility. Another option would 
be to develop a new separate part that distinguishes the purposes of compliance, es- 
pecially in relation to agency reviews done at the European level in which the ESG are 
used as a direct reference tool. Or perhaps, another suitable suggestion would be to 
elaborate a guide on how to use the ESG for external reviews or an explanatory report 
similar in approach to that of the Lisbon Recognition Convention 3 . 

UNDERSTANDABILITY OF THE ESG 

In order for the ESG to achieve the stated purposes it is essential not only to create 
awareness and knowledge, but also for the ESG to be easily understandable to higher 
education stakeholders active at the European level, but more importantly also to 
those active nationally and at an institutional level. One of the purposes of the stand- 
ards and guidelines addresses explicitly the importance of understandability of the 
ESG by stating that their purpose is to make external quality assurance more transpar- 
ent and simpler to understand for everybody involved (esg, p. 14). Furthermore, to 
achieve the objectives of the ESG, the ESG need to be understandable to »higher edu- 
cation institutions, students, employers and other stakeholders« (esg, p. 14). Moreo- 
ver, the ESG clearly state that one of the objectives is »to contribute to a common 
frame of reference for the provision of higher education and the assurance of quality 
within the ehea« (esg, p. 14), which also can only be achieved when there is a com- 
mon understanding of what is written in the ESG 

Given the diversity of national and institutional contexts, as well as the range of 
stakeholders involved in quality assurance, and indeed the acknowledged role the 
ESG should play in the wider societal context, for the ESG to achieve the set purposes 
and objectives the understandability of the ESG is of paramount importance. The ESG 


3 For further information on the Lisbon Recognition Convention, including the explanatory 
report, see http://www.coe.int/t/dg4/highereducation/recognition/lrc_EN.asp 
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themselves also address the complexity of achieving joint understanding. The intro- 
duction to the ESG acknowledges the differences in the understanding of the same 
quality assurance terminology in different national and institutional contexts, but 
also the differences in perspectives between the different stakeholders. 

For the ESG to be easy to understand for everybody involved, clarity is essential. How- 
ever, clarity itself is not a simple term. Firstly, the language used, particularly the ter- 
minology needs to be easy to understand. Secondly, the interpretation of standards 
and guidelines needs to be unambiguous. Thirdly, the understanding of multiple in- 
tentions and purposes of the ESG need to be as easy to understand as the language and 
terminology, in order to provide the essential framework for the interpretation of the 
ESG in different contexts. In this Section clarity of language, particularly the terminol- 
ogy and the ambiguity of the ESG are addressed; intention and purposes of the ESG 
have been addressed in Section 1. 

The results of the analysis of the survey of national unions of students show that the 
majority of the respondents find the ESG as a whole to be mostly understandable, with 
some parts being flagged as not understandable. One of the issues that national unions 
identify as a reason that hinders easy understanding and clarity of the ESG is the lan- 
guage, phrasing and in particular the meaning of the specific quality assurance terms 
used in the ESG. Unlike the national unions, the individual students who took part in 
the international reviews of quality assurance agencies regard the language of the ESG 
as relatively easy to understand. 

One of the reasons maybe the difference in the exposure to international quality assur- 
ance language. While the individual students who took part in external quality assur- 
ance reviews have been taking active part in international events dealing with higher 
education and quality assurance issues, only a few national union representatives have 
opportunities to take part in the international discussions. This is even more the case 
for local unions of students. As one of the national unions elaborates »Local unions 
are not used to international qa language, that’s why it’s hard to see connections be- 
tween international standards and higher education institutions’ regulations 

While the individual students who took part in the external reviews of quality assur- 
ance agencies assessed that the language and the terminology of the ESG was rather 
clear, they explicitly stated for most of the standards that they cannot be interpreted 
in an unambiguous manner. This indicates that even though these student reviewers 
may have a better understanding of the international QA language the ESG remain dif- 
ficult to interpret in international external reviews. 

The terminology used in the ESG is not only international in its nature; certain terms 
are also used inconsistently throughout the ESG. For instance the ESG refer to their 
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purposes, objectives and aims throughout the introductory chapters without clari- 
fying what the difference is between these three terms, and in some instances these 
terms do seem to be used interchangeably. Similarly, terms like student participation 
and student involvement, as well as stakeholder participation and stakeholder involve- 
ment are also used interchangeably throughout the ESG, although they may have dif- 
ferent meaning and connotations nationally and at institutional level regulations and 
practice. This difference that might exist between understanding the ESG on one hand 
and interpreting them consistently on the other also became apparent in the focus 
group. Whereas most participants indicated that they had no difficulties in under- 
standing the ESG, when further discussing some of the key terms used it became clear 
that there was a wide variety in interpreting what these terms actually meant to them. 

The discussions in the focus group and the text-analysis of the ESG showed that the 
inconsistencies in the way in which different standards and guidelines are written con- 
tributes to ambiguities and makes the ESG more difficult to understand. Students in 
the focus groups, as well as the national unions of students noticed that some stand- 
ards are written in a more precise way than the others. 

Apart from the precision of formulations in individual standards and guidelines, a 
careful textual analysis of the ESG suggests that there are standards which may be 
vague or which are not elaborated much, while their accompanying guidelines are 
very concrete and bring in important elements for implementation and guidance. 

For instance, the standard 1.3 Assessment of students in itself is rather brief and seem- 
ingly to the point, whereas it’s accompanying guideline brings numerous important 
elements which clarify in detail what student assessment procedures are expected to 
be. On the other hand, the standard 1.5 Learning resources and student support is 
formulated in a manner open to interpretation, stating merely that institutions should 
ensure that resources available for the »support of student learning are adequate and 
appropriate« ( esg , p. 18, emphasis added by authors). Unlike the guideline accompa- 
nying the standard 1.3 Assessment of students, the guideline accompanying the stand- 
ard 1.5 Learning resources and student support is formulated in a similar manner that 
leaves much to interpretation. 

These inconsistencies are also visible in standards 1.1. Policy and procedures for quality 
assurance, 1.2 Approval, monitoring and periodic review of programmes and awards, 
2.4 Processes fit for purpose, 3.6 Independence, 3.8 Accountability procedures, where 
the guidelines also bring in additional important elements to the standards, while 
standards 1.4 Quality assurance of teaching staff, 2.1 Development of external quality 
assurance processes, 2.3 Criteria for decisions, 2.6 Follow up procedures, 3.1. Use of 
external quality assurance procedures for higher education, 3.3. Activities are formu- 
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lated in a manner open to interpretation and the accompanying guidelines don’t nar- 
row the scope of possible interpretation. 

The national unions pointed out that they have more difficulties with clarity of the 
standards and guidelines in Part 1: European standards and guidelines for internal 
quality assurance within higher education institutions, compared to standards and 
guidelines in Part 2: European standards for the external quality assurance of higher 
education and Part 3: European standards for external quality assurance agencies. This 
is further addressed in the part on the relation between standards and guidelines. 

Although, for each of the standards in all three parts of the ESG national unions gave 
divergent views on their clarity, some standards can be highlighted as less understand- 
able than others. With regards to the Part 1 of the ESG standards 1.1 Policy and proce- 
dures for quality assurance and 1.4 Quality assurance of teaching staff were identified 
as those which were more difficult to understand compared to other standards. With 
regards to Part 2, this seems to be the case particularly with standard 2. 2. Development 
of external quality assurance processes, 2.4 Processes fit for purpose, 2.5 Reporting 
and 2.8 System-wide analysis. Interestingly, these are also the standards the national 
unions of students identified as less often implemented consistently in the national 
systems (See Section 3). 

The individual students who had been involved in qaa reviews particularly pointed 
out that the standard 2.1 was virtually never used in the external reviews, indicating 
that the standard may not be clear nor easy to understand and be used in an unam- 
biguous manner in the external reviews. 

Having in mind the importance of ESG to be easy to understand for everybody in- 
volved the results of the esu consultation clearly show the necessity for the ESG to be 
improved with a view to achieving greater clarity of the language and terminology 
used, but, as importantly, for addressing the ambiguity of the ESG. The national unions 
clearly stated that enhancing the clarity of standards, next to further elaborating and 
developing the ESG, was the most important way in which the ESG should be revised. 
This was also supported by the focus groups. 

While addressing the clarity of language and terminology may be tackled through 
elaborating further the differences between national contexts and approaches to qual- 
ity assurance, as in the introductory parts of the ESG, and through the creation of a 
glossary of terms placed in the context of the ESG, the issue of ambiguity may be more 
complex to address. However, it is essential to address in order to enable consistent use 
of the ESG in the construction of national or regional standards or in external reviews 
of both higher education institutions and quality assurance agencies, as shown below. 
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Logic of the ESG and the relationships between different parts of the ESG and the 
Standards and the Guidelines 

The ESG are divided into three parts. Part 1 addresses internal quality assurance, Part 2 
external quality assurance and Part 3 addresses the functioning of qaas. The different 
parts are linked through standards 2.1 Use of internal quality assurance procedures 
and 3.1 Use of external quality assurance procedures for higher education respectively. 
According to this logic, external quality assurance of higher education institutions 
needs to take into account the provisions of internal quality assurance due to the link 
of standard 2.1 to Part 1 of the ESG. Similarly, the external quality assurance of qaas 
needs to take into account how the agency evaluates higher education institutions 
or programmes in external reviews due to the link of standard 3.1 to Part 2 of the ESG. 
Thereby via standard 2.1 also Part 1 of the ESG is linked, meaning that in an external 
evaluation of a quality assurance agency it must also be reflected how the agency en- 
sures that the higher education institutions have internal quality assurance systems 
compatible with the standards and guidelines on internal quality assurance (part 1 of 
the esg). 

It could be assumed that the main emphasis of reviews always rests on the part of the 
ESG that is most visibly related. This does, however, not mean that the other part(s) of 
the ESG can be disregarded since through the logic of the standards all three parts are 
linked. 

The majority of respondents who have undertaken reviews of qaas report that also the 
standards of Part 2 have been taken into account, thus the link between Part 2 and Part 
3 of the ESG seems to work. However, some also report that Part 2 has been neglected 
or has not adequately been taken into account. The standards and guidelines of Part 1 
only an alarmingly few respondents report as having been properly taken into account 
and there have even been several respondents who have reported that Part 1 has been 
entirely neglected. 

This poses the question of whether the intended linkage between internal quality as- 
surance and external quality assurance functions as it is clearly contestable how rel- 
evant and important the linked parts of the ESG are deemed in cases where the main 
focus is placed on another part. It is evident that there is no consistency in this matter 
but that the intended connection between the different parts of the ESG is only par- 
tially implemented. 

The majority of respondents indicate that both standards and guidelines are generally 
being taken into account during reviews of the qa Agencies. With some standards, 
the guidelines are always taken into account whereas some say that the guidelines for 
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Standard 3.7 External quality assurance criteria and processes used by the agencies 
have not or only in parts been taken into account. 

When analysing the ESG, an apparent issue is why some elements are part of the stand- 
ards and some are part of the guidelines. For example, Standard 2.4 has an elaborate 
and fairly detailed list of elements in the guidelines, whereas Standard 3.7 defines ele- 
ments much more clearly in the standard itself. 

Also respondents indicate that there is no clarity as to what is the difference between 
standards and guidelines. Most argue that guidelines are essential in order to clarify 
the standards, which at times are more difficult to understand. Also a large number of 
respondents say that the guidelines add valuable information. Only few view some of 
the guidelines to contain unnecessary repetitions of what is already in the standards. 
These and other inconsistencies with the way in which the standards and guidelines 
are being formulated have been further elaborated in the sub-sections on Scope, Un- 
derstandability and Impact. 

Also respondents to the qaa reviewers survey and students in the focus group have 
indicated that there is no clarity as to what is the difference between standards and 
guidelines. Most argue that guidelines are essential in order to clarify the standards, 
which at times are more difficult to understand. Also a large number of respondents 
say that the guidelines add valuable information. Only few view some of the guide- 
lines to contain unnecessary repetitions of what is already in the standards. 

Some respondents would like to keep the separation between standards and guidelines 
whilst others would like the guidelines to be incorporated into the standards. Based 
on the responses from the national unions and the members of the esu qa expert pool, 
there is a tendency that the incorporation into the standards is more sought for in rela- 
tion to Part 1 of the ESG than in relation to Parts 2 and 3. The explanations given for 
why guidelines should be incorporated into the standards are mainly that this would 
clarify the standards and make them easier to understand, or that they would be easier 
to check in external reviews. However, only few say that otherwise the standard would 
be misunderstood or not be implemented at all. Even fewer say that else the elements 
contained in the guidelines would not be checked in external reviews at all. 

Respondents have also indicated that according to their experience, reviewpanel mem- 
bers often deem standards to be of different importance. Some of the respondents also 
support this view whilst others believe all standards should be given the same impor- 
tance. Although there is variation, review teams of quality assurance agencies seem to 
give higher importance to Standards 3.1 and 3.6 and low importance to Standard 3.8. 
The experience of the respondents might be specific to their review teams. However, 
there is a clear tendency that not all standards are given the same importance and that 
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there can be quite some deviations with regard to which standards are prioritised over 
others, depending on individual opinions. 

Again, this shows that there is no consistency in the application of the ESG and hence 
reasons to contest equality and reliability of reviews. This might also be related to 
the difference in clarity and preciseness of standards in comparison to each other. In 
addition, some standards and guidelines are much more elaborate and detailed than 
others. Some provide much more room for interpretation than others. This might be 
due to different views regarding how detailed and prescriptive the ESG should be, but 
does not explain the reason why there is this inconsistency between the standards. It 
might be beneficial to discuss whether or not some standards are deemed more im- 
portant than others and should play a more prominent role in reviews. This becomes 
even more important concerning the judgment of compliance with the ESG generally; 
i.e. are there core standards for which compliance is indispensable in order to comply 
with the ESG as such? Similarly, even if all standards would be seen as equally impor- 
tant, the question remained as to whether all of them require compliance in order to 
achieve overall compliance with the ESG. 
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SECTION 3: IMPLEMENTATION AND 
IMPACT OF THE ESG 


IMPACT AND IMPLEMENTATION OF THE ESG 

Although carefully balanced with the autonomy of higher education institutions, the 
European Standards and Guidelines aim to have an impact on internal and external 
quality assurance processes of higher education institutions and on national quality 
assurance systems as a whole. 

At the time of adopting the ESG (May 2005) a number of countries participating in the 
Bologna Process were embarking upon the development of their quality assurance 
systems, while a number of countries had quality assurance systems already in place. 
According to the data gathered from the national unions of students in the countries 
where the quality assurance systems were developed after the adoption of the ESG, the 
ESG played an important role in providing guidance for the development of internal 
and external quality assurance procedures, as well as in setting up the quality assur- 
ance agencies. Furthermore, the national unions of students of countries where qual- 
ity assurance systems were already in place before the adoption of the ESG report that 
when reforms or changes of these quality assurance systems happened after 2005, the 
ESG were taken into account. Even though it is most likely that the impetus for de- 
veloping quality assurance systems came from the Bologna Process commitments in 
general, in particular those related to qa, and changes to existing quality assurance 
systems could have been initiated nationally for a variety of reasons, it is clear that 
the ESG had a considerable impact on the national qa systems since their adoption, 
at least in the form of providing guidance. Nevertheless, in some cases, as reported 
by national unions, the ESG were followed in detail when creating or changing the 
national quality assurance systems. 

However, this wide use of the ESG in the development and changes of the quality as- 
surance systems does not imply that the standards and guidelines have been imple- 
mented consistently. The analysis of the information received from the national and 
local unions suggests that all of the quality assurance systems and most of the heis in 
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the sample have similar standards to those defined in the ESG as part of their systems, 
but at the same time they have not (yet) implemented some important elements of the 
standards or some standards in their entirety. 

Holistically, the data gathered about national quality assurance systems suggests that 
national guidance/regulations for internal quality assurance within higher education 
institutions are often in line with the standards in Part 1 of the ESG. But according to 
data obtained from students on the institutional level, this does not necessarily mean 
that these are also implemented within heis. This suggests that there might be a miss- 
match between national legislation on qa, mostly corresponding to the ESG, and its 
implementation within institutions. 

At the same time the national standards for the external quality assurance of higher 
education are less often seen as being identical or similar to those defined in Part 2 of 
the ESG; most often the national standards differ in important elements from the cor- 
responding standards in the ESG and some ESG standards are not part of the national 
quality assurance systems at all. This observation from national unions of students 
and the participants in the focus group may in part be due to them being more familiar 
with the internal quality assurance procedures in heis and it may also suggest that 
standards for the external quality assurance of higher education are less available to 
the academic community, as well as the general public. As mentioned in Section 2, a 
third of the students who took part in the external reviews of qaas report that the 
standards in the Part 1 are disregarded in the external reviews, while Part 2 and Part 3 
are according to the majority of respondents taken into account, however not always 
properly. 

As mentioned earlier in Section 2 of this report, all of the standards related to exter- 
nal quality assurance of higher education are less often defined nationally in a similar 
manner to Part 2 of the ESG, but this seems to be the case particularly with Standards 
2. 2. Development of external quality assurance processes, 2.4 Processes fit for purpose, 
2.5 Reporting and 2.8 System-wide analysis. 

With regard to standards for external quality assurance agencies (esg Part 3), the 
Standards 3.2 Official status, 3.4 Resources and 3.6 Independence seem to be most 
often not defined in national systems or with important elements missing. 

We will now look a bit further into the way quality assurance is functioning on the 
institutional level, possibly the most important level in terms of quality assurance 
of higher education. Given the scope of this report, our interest lies primarily in the 
question to what extent internal qa within heis is consistent with Part 1 of the ESG 
containing the European standards and guidelines for internal quality assurance 
within higher education institutions. 
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The first three standards and their accompanying guidelines of Part 1 refer to policy 
processes and mechanisms of quality assurance of heis with regard to their study pro- 
grammes and rewards, with particular attention for elements related to the quality 
assurance of education and teaching (e.g. curriculum design; development of learning 
outcomes; student assessment). 

Compared with most of the other Standards of Part 1 of the ESG, Standard 1.1 Policy 
and procedures for quality assurance and its guidelines provides a relatively broad 
overview of aspects that are expected to be part of a heis internal qa system. It de- 
scribes not only what qa policies and procedures are supposed to be (»a framework 
within which higher education institutions can develop and monitor the effectiveness 
of their quality assurance systems «), but also what the underlying aim of such policies 
and procedures would be (in particular »the development of a culture which recog- 
nises the importance of quality and quality assurance, in their work«; » continuous 
enhancement of quality«; helping in providing »public confidence in institutional 
autonomy«). These aims of qa processes and procedures are of course partly a reflec- 
tion of some (but not all) of the aims/ purposes that are mentioned in the introductory 
chapters to the ESG (and further discussed in Section 1 of this report). 

But besides an explanation of internal QA policies and their aim, standard 1.1 also men- 
tions a few concrete areas that are expected to be included in the qa policy statement 
of every hei, including the relationship between teaching and research; the respon- 
sibilities of the different organisational levels within a hei (e.g. departments, schools, 
faculties); and the involvement of students in QA. 

Also the focus group pointed out that whereas the organisation of an internal qa sys- 
tem is described quite extensively in this standard, some of the abovementioned aims 
and >additions< are not always clear or on the right place in this Standard. 

Given this wide range of elements, explanations and recommendations that are in- 
cluded in Standard 1.1, it might not come as too large a surprise that, according to 
national unions of students, important elements of Standard 1.1 (or in some cases even 
the Standard as a whole) are not part of the national regulation on qa. 

When looking at the actual implementation of Standard 1.1 within heis, a similar pic- 
ture appears. Almost all the respondents to the local student representative survey 
answer positively when asked if their hei has internal policies and procedures for qa. 
These policies and procedures are generally publicly available and are in their opinion 
aimed at a continuous enhancement of higher education. 

So if we would only look at the title of Standard 1.1, the conclusion would be that based 
on our data, heis tend to have policies and procedures for quality assurance in place 
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and thus operate in accordance with this standard. But that is only part of the truth. 
The positivism of local student representatives about the existence, availability and 
main aim of the internal QA policies and procedures goes hand in hand with diverse 
views about the appropriateness of the division of responsibilities for qa between dif- 
ferent organisational levels, and negative opinions about the way in which the relation- 
ship between teaching and research is being ensured and improved, and in particular 
about the level of involvement of students and other stakeholders in qa. 

Interesting enough, the apparent weakness of these last two areas also came out of the 
focus group when they did a textual analysis of Standard 1.1. There was a strong con- 
sensus that in particular the relationship between teaching and research and the role 
of students and other stakeholders appear to be somewhat tokenistic and need much 
more emphasis and clarification. Partly, this has also been addressed in previous sec- 
tions of this report and we will pick up the particular point about the role students and 
other stakeholders in the next chapter. 

A majority of the national unions of students report that all the elements of Standard 
1.2 Approval, monitoring and periodic review of programmes and awards can be found 
in the national QA system, whereas a quarter says that important elements are missing 
in the national QA system. 

The vast majority believes the standard should not be amended and they also remark 
that the guidelines are both a good clarification of the standard and add vital informa- 
tion to it without providing unnecessary repetitions. However, the majority of them 
say that the guidelines of parts of them should be integrated into the standard for rea- 
sons of clarification and avoiding misunderstandings, easing external evaluations and 
to ensure implementation. 

The vast majority of respondents from the local level say that heis have implemented 
the standard and also most of them remark that the guidelines are being followed with 
the notable exception of the gathering of feedback from employers and other stake- 
holders, which about half the respondents say is not done. 

The majority of the national student unions considers Standard 1.3. Assessment of stu- 
dents very relevant to the functioning of heis. Only few unions believe that the stand- 
ard should be amended when asked directly, however, when asked if the guidelines 
should be integrated into the standard or if parts of the guidelines should be half of the 
unions agrees that indeed the vital elements of the guidelines should be added to the 
standard. Furthermore, those who do believe that the standard should be amended 
as such specify that the assessment of students should be designed to measure the 
achievement of intended learning outcomes. 
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This is in line with the outcomes of the focus group. The students in the focus group 
also thought that vital elements of guidelines should become a part of the standard, 
but more importantly they believed that a clear link between the student assessment 
and learning outcomes was missing both in the standard itself and its accompanying 
guidelines. 

Interestingly, the local unions report that they do believe that there are institutional 
or departmental regulations on the student assessment within their higher education 
institutions, but at the same time report that they are not familiar with them partly be- 
cause they are not publically or widely available. Furthermore, barely any local union 
reported that student assessment is intrinsically linked to learning outcomes. 

The data, thus, suggests that further work is needed to truly implement the concept 
of learning outcomes into the daily practice of higher education institutions, which 
seems to be of high importance for students and student representatives alike. 

Part 1 of the ESG also pays attention to what can be considered preconditions to a good 
process of education. This covers a wide range of areas, including qa of teaching staff 
(Standard 1.4); learning resources and student support (Standard 1.5), information 
systems (Standard 1.6); public information (Standard 1.7). 

The ESG do not specify why certain preconditions have their own standards whereas 
others are merely mentioned in the explanatory text to the standards (e.g. stakeholder 
involvement) or only referred to in the guidelines (e.g. teaching-research links). These 
examples are not chosen at random, but were explicitly mentioned in the focus group 
as topics that deserve more attention and emphasis in the standards and guidelines 
of (in particular) Part 1 of the ESG. The issue of stakeholder involvement is further 
covered later. Concerning the relationship between teaching and research it needs 
mentioning that besides the focus group, also the local student representative sur- 
vey suggests that this is an area that might need more attention, as a large majority of 
the respondents indicate that they either do not know how the relationship between 
teaching and research is being quality assured or report that the policies and proce- 
dures in place are not adequate. 

Notwithstanding the question why certain preconditions for quality of higher educa- 
tion have been introduced as standards and others not, the fact that the standards 
which do contain preconditions for quality are formulated alongside standards that 
are more directly related to policies, procedures and mechanisms of qa (Standards 
1.1, 1.2 and to some extent 1.3) clarifies that these are equally important to take into 
account for heis in assuring and enhancing quality of education. In other words, the 
ESG suggest that good quality education is not just dependent on proper evaluation 
and improvement mechanisms, but also on a (minimum) number of preconditions 
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that need to be satisfied to enable teaching and learning to be provided in the first 
place. For example, Standard 1.6 and its Guidelines stipulate that heis are expected 
to provide Public Information in a certain way (impartial, objective, readily acces- 
sible, quantitative and qualitative, about programmes and awards etc.), rather than 
suggesting that Public Information is one of the areas that needs to be evaluated and 
quality assured. 

An exploration of these elements of QA through analysis of the surveys at local and 
national level shows an interesting and diverse picture of the extent and way in which 
the preconditions to good quality education, as outlined in the ESG, have been imple- 
mented within HEIS. 

Standard 1.4 Quality assurance of teaching staff was identified by national unions of 
students as the one where some important elements of the standard were not part of 
the national regulation or were not part of them at all. 

The local student survey shows that the majority of local student representatives are 
not (fully) aware of the mechanisms their hei has in place to make sure that staff in- 
volved with the teaching of students are qualified and competent in the subject they 
are teaching and have the necessary skills and experience to transmit this effectively 
to students. Because of this lack of awareness, arguably due to a low level of involve- 
ment of student representatives in issues related to teaching staff (see more on stake- 
holder involvement later), it is not possible to draw clear conclusions about the extent 
to which the staff procedures as outlined in the guidelines to Standard 1.4 are actually 
being implemented. 

It is worth noting, however, that local student representatives that are aware of staff 
procedures in their hei are largely positive about the opportunities that exist for train- 
ing and development of teaching staff (including those who have a poor performance), 
whilst being sceptical about the possibilities their hei have to remove staff members 
from their teaching duties if they continue to perform poorly and are demonstrably 
unsuitable to teach. 

Contrary to procedures connected to QA of teaching staff, student representatives that 
responded to the surveys generally have a good awareness of the ways in which their 
hei tries to ensure that the resources available for the support of student learning 
are adequate and appropriate for each programme offered, i.e. Standard 1.5 Learning 
resources and student support. According to the vast majority of respondents, their 
heis indeed have mechanisms in place to routinely monitor, evaluate and improve 
the effectiveness of the support services available to students. However, less than half 
of them believe that this is done in a sufficient way. Here it should be noted that there 
is a slightly higher satisfaction about the way quality assurance and enhancement of 
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physical learning resources (such as libraries or computing facilities) are function- 
ing than that of human resources for the support of student learning (such as tutors, 
counsellors and other advisors). 

When it comes to standards 1 .6 Information systems and 1.7 Public information, a con- 
siderable amount of national unions pointed these out as being not fully implemented 
in the national quality assurance systems. 

The vast majority of local student representatives have indicated that the systematic 
collection, analysis and use of information that is relevant for the effective manage- 
ment of the study programmes and other university activities is indeed part of the 
qa activities of their hei. The quality-related information systems required by heis 
do however not always cover all the elements that are expected in the guidelines to 
Standard 1.6. According to local student representatives, heis make good use of data 
connected to student progression and success rates; students’ satisfaction with their 
programmes and the hei’s own key performance indicators. On the other hand, espe- 
cially relevant data on the profile of the student population; availability of adequate 
learning resources; effectiveness of teachers; and employability of graduates is in 
many cases not collected and/or not used in the management of student programmes 
and other university activities. 

Compared with most other standards in the ESG, Standard 1.7 Public information is 
relatively directive. It does not say that qa mechanisms should be in place to ensure 
that heis provide public information about the programmes and awards they are of- 
fering (which would be more in line with the previous standards) but insists that insti- 
tutions should publish up to date, impartial and objective information. This suggests 
that the ESG considers the provision of up to date, impartial and objective information 
a universal prerequisite to qa of higher education and in the surveys we have asked 
student representatives how this is being implemented in their local and national con- 
text. 

Almost all the surveyed local student representatives confirm that some information 
about the programmes and degrees their hei offers is publicly available. This informa- 
tion tends to be up to date and includes most of the elements suggested in the guide- 
lines to Standard 1.7, related to the content and structure of the study programmes and 
teaching and learning processes. 

On the other hand, information that could be as relevant for the wider public (includ- 
ing future students and their parents) related to for example the profile of the current 
student population; views of past students about their educational and learning expe- 
rience; and employment destinations of past students is, according to the respondents, 
in most cases not provided by heis. 
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Furthermore, although most local student representatives believe that the informa- 
tion their heis provide to the wider public is impartial and objective, they also report 
that this information is more often primarily intended as a marketing tool than as a 
way of objectively informing (future) students and the wider public. 

An analysis of the data collected from national unions and local student representa- 
tives, paints a very diverse picture of the way and extent to which standards and guide- 
lines are being implemented within national higher education systems and heis. 

Whereas national unions of students already indicate that not all standards and guide- 
lines have had a similar impact on the national quality assurance systems, a closer 
look at the institutional level shows that in all of the countries in our sample, as di- 
verse as they are, there are heis with a system of internal qa in line with the ESG, but 
more heis that have not. The responses from local student representatives suggests 
that the implementation of a qa system that follows the principles of the ESG relies 
more on heis than on national governments, and means that it is possible that where 
in one institution certain standards are almost completely translated into the internal 
QA system, another hei in the next town might have a system in which a number of 
standards and guidelines are lacking. 

Standards that stand out in particular in this respect are Standards 1.5 Learning re- 
sources and student support, 1.6 information systems and 1.7 Public information 
which in many cases are not fully implemented in heis which might have a connection 
with observations of a considerable number of national unions who indicate that these 
standards are not fully implemented in the national quality assurance systems either, 
which could have provided a stimulus and guidance to heis for a more complete im- 
plementation of this standard. 

A more nuanced picture exists of Standard 1.4 QA of teaching staff, which has also of- 
ten been identified as a standard, which is not or only partly implemented in national 
regulation. With regards to this standard, students show a lack of awareness on one 
hand and scepticism about existence and functioning of staff procedures. 

The way in which guidelines to Standard 1.4 qa of teaching staff are formulated, sug- 
gests that adequate staff procedures are seen as an essential element of the assurance 
of the quality of teaching staff and unambiguously stipulate what heis are expected 
to include in the procedures regarding staff recruitment, appointment and improve- 
ment. Although fully agreeing with the importance of this standard and accompany- 
ing guidelines, the focus group of students of the esu qa experts pool questioned the 
limitation an emphasis on quality of teaching staff could have, since in their opinion 
also the quantity of teaching (and support) staff ought to be an element in standards 
and guidelines for qa. 
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The revision of the ESG should not be taken lightly. Furthermore, further revision of 
the ESG should take into account that the ESG are indeed used nationally at least as 
guidance in developing quality assurance systems, and not only internationally for 
the reviews of quality assurance agencies. The insight into inconsistencies with the 
implementation of the standards suggests that the revisions are indeed necessary to 
ensure greater clarity of the intentions and purposes of the individual standards and 
guidelines. 


STAKEHOLDER INVOLVEMENT 

In the introductory chapters of the ESG document, the importance that qa has for 
students and other stakeholders is explained in one of the basic principles underlying 
the ESG. This principle argues that »the quality of academic programmes need(s) to 
be developed and improved for students and other beneficiaries of higher education 
across the ehea« (esg, p. 14 ). Also, »the interests of students and other stakeholders 
such as labour market representatives should be at the forefront of external quality 
assurance processes« (esg, p. 15 ). 

Furthermore, one of the stated objectives of the standards and guidelines is »to in- 
form and raise the expectations of higher education institutions, students, employers 
and other stakeholders about the processes and outcomes of higher education« (esg, 
P-h)- 

The importance that quality assurance has for a wide range of stakeholders has been 
exemplified in the Berlin Communique, where ministers invited enqa, esu, eua, and 
eurashe to develop »an agreed set of standards, procedures and guidelines on qual- 
ity assurance (...) and to report back through the Bologna Follow-Up Group to Min- 
isters in 2005« (Berlin Communique, p. 3). Following this mandate, these organisa- 
tions, representing different key stakeholders, worked together to produce the actual 
standards and guidelines. 

Although the ESG have been written in the spirit of cooperation between stakehold- 
ers and the interests of stakeholders are mentioned explicitly in the introductory 
chapters of the ESG document, it is interesting to observe that the principle of stake- 
holder involvement does not come back largely in the actual standards and guidelines. 
Whereas involvement of students still features in the text of Standard 1.1 and 3.7 and 
the guidelines falling under Standards 1.1, 1.2, 1.5, 2.4, >other stakeholders< are mostly 
referred to as a wide and undefined group and the scope and degree of their expected 
involvement in qa is not made very explicit. Standard 1.1 stipulates that »the strategy, 
policy and procedures for qa [of heis] (...) include a role for students and other stake- 
holders« (esg, p. 16). 
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But what this role should be and who these other stakeholders might be, if this refers 
in particular to internal (e.g. teaching staff) or external (e.g. employer and employee 
organisations) stakeholders is not made clear. The guidelines to Standard 1.2, however, 
are more specific. As part of a rather long list of other points, the qa of programmes 
and awards are expected to include »regular feedback from employers, labour market 
representatives and other relevant organisations« (esg, p. 17). Although not explicitly 
mentioned, it is logical to assume that all of these are expected to be external to the 
hei and do not refer to internal organisations like for example staff organisations. In 
this respect it is interesting to notice that whereas there is an entire standard devoted 
to the quality assurance of teaching staff, no clear reference is made in the ESG about 
their involvement in quality assurance. 

But although the expectation of obtainment of regular feedback from external stake- 
holders as part of qa of programmes and awards is included in a guideline, only a third 
of the local student representatives answered confirmative when asked if this is actual 
practice within their hei. 

During one of the sessions of the focus group, the students of the esu QA experts’ pool 
undertook a critical analysis of the way in which the standards and guidelines in the 
ESG are formulated. During that process, it became immediately apparent how the 
topic of student involvement was referred to in a seemingly ambiguous, or, as some 
participants preferred to call it, >vague< way. 

In Part 1 of the ESG there are three sets of standards and guidelines that refer to stu- 
dent involvement. Standard 1.1 Policy and procedures for quality assurance mentions 
that there should be a role for students in the hei’s strategy, policy and procedures for 
qa. But what this role should be is not very clear. It could equally be one of an active 
participant in all aspects of qa as that of a passive recipient of information about the 
results of information about the final outcomes of qa processes. The >role of students< 
becomes only slightly more concrete in the guidelines to Standard 1.1, where it is indi- 
cated that the hei’s policy statement to qa »is expected to include the involvement of 
students in quality assurance« (esg, p. 16), suggesting at least some sort of active, yet 
still not further defined, role of students. 

Arguably, for most people >involvement< would have a similar meaning as participa- 
tion^ a term used in the guidelines to Standard 1.2 Approval, monitoring and periodic 
review of programmes and awards. Where in the guidelines to Standard 1.1 students 
are expected to be involved in quality assurance in general, according to the guide- 
lines to Standard 1.2 the quality assurance of programmes and awards are »expected 
to include participation of students in quality assurance activities«. The term par- 
ticipation of students< also comes back in the guidelines to Standard 2.4 Processes fit 
for purpose, here referring to external review processes. 


44 


ESU CONSULTATION REPORT OF THE MAP-ESG PROJECT 



In contrast to this broad and generic use of the term student involvement/partici- 
pation, the guidelines to Standard 1.5 Learning resources and student support and 
Standard 3.7 External quality assurance criteria and processes used by the agencies 
use much more explicit terminology for the role students are expected to play in these 
aspects of qa. The guidelines to Standard 1.5 suggest that heis should seek feedback of 
students about learning resources and other support mechanisms and be >responsive< 
to this. Standard 3.7, on the other hand, uses arguably the most concrete terminology 
on student participation to be found in the ESG, by recommending that the processes 
used by external quality assurance agencies »will normally be expected to include 
an external assessment by a group of experts, including, as appropriate, (a) student 
member(s) (...)« (esg, p. 25 ). However, although the inclusion of a student member 
in an external review panel is indeed a pretty straightforward and clear recommenda- 
tion related to student involvement, in the implementation of this standard there is 
definitely space for confusion and differences in interpretation about when a student 
member might not be appropriate and why it was deemed necessary to bring such a 
nuance to this particular part of Standard 3.7. 

From this textual analysis of the ESG it becomes clear that regardless whether you refer 
to it as student involvement or student participation, or even student feedback, stu- 
dent engagement (yet another term often used) in qa can mean many different things 
to different people. With regard to internal qa within a hei, student participation can 
for example refer to students as providers of information and feedback, who are mere- 
ly asked to fill in surveys about their modules and study programmes; to students as 
actors in qa who have influence on the collection and analysis of student feedback; or 
to students as full partners in university decision-making processes, including those 
related to qa (c/. enqa, 2006 ; Heerens, 2010 ). 

The different interpretations of what is meant by student participation is partly a re- 
flection of the diversity and autonomy of heis, qaas and national systems of higher 
education and therefore it is not surprising that the ESG do not define and explain it 
further. However, when analysing the ESG as a whole, it also becomes clear that there 
is no real consistency in the level of concreteness used for different elements of qa that 
are mentioned in the standards and guidelines. Some have been made into rather con- 
crete and explicit standards and guidelines, like for example assessment of students 
(Standard 1 . 3 ) or information systems (Standard 1 . 6 ), which give through the respec- 
tive guidelines a detailed overview of minimum requirements heis have to take into 
account and are therefore supposedly universally applicable throughout the ehea. 

As, in the words of former enqa President Peter Williams, »in recent years, the role of 
students in the quality assurance of higher education has become recognised, across 
Europe, as being both necessary and desirable« (enqa, 2006, p. 5) and the Berlin 
Communique already specifically argued for the inclusion of student participation in 
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the quality assurance systems of all countries participating in the Bologna Process 
(Berlin Communique, 2003) one might expect that a more specific and detailed refer- 
ence to student participation would be included in the ESG. Such a reference would not 
have to be limited to student involvement in the external quality assurance processes 
of quality assurance agencies, but also in relation to both internal quality assurance of 
heis and external quality assurance of heis or study programmes. Perhaps this could 
even take the form of a separate standard. Interesting enough, at exactly the same 
occasion as the presentation of the ESG, during the Ministers Conference in Bergen 
in 2005, a Stocktaking report on the progress made in three priority action lines — of 
which quality assurance was one — was presented, in which student participation in 
qa was to some extent described more specifically than in the ESG. In this report, stu- 
dent participation was expected to relate to (at least) four different levels: 

O In the governance of national bodies of QA; 

O Within teams for external review; 

O Consultation or involvement during external reviews; 

O Involvement in internal evaluations; 

(Bologna Process Stocktaking, 2005, p. 17). 

Besides the issue of clarity about what is exactly meant by the different terms related 
to student participation that are used in the ESG, almost all the participants of the 
focus groups failed to understand why a form of student involvement features in only 
a few of the standards and guidelines, and not in others. Especially in Part 1 of the ESG 
it appeared difficult to find a logic behind the fact that student participation features 
in Standards 1.1, 1.2 and 1.5, but not in the others. Certainly, the omission of student 
participation in Standards 1.3, 1.4, 1.6 and 1.7 does not necessarily mean that the ESG 
recommend that students have no role in qa related to, for example, student assess- 
ments or teaching staff. But when countries and heis implement the ESG, the impres- 
sion might still occur that student participation is only relevant and to be encouraged 
in areas of qa where it is explicitly mentioned. 

Data obtained through the local student representative survey seems to support this 
premise. Although according to the respondents, in most heis there are clear regula- 
tions about student involvement in quality assurance, the number of respondents who 
indicate that the level of involvement is not sufficient is twice the number of those who 
are satisfied with it. Furthermore, a large minority of respondents, spread over the dif- 
ferent countries in the sample, indicate that there are no clear regulations on student 
involvement in qa at all. 
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In the survey, local student representatives have been asked how much they are in- 
volved in the areas of QA covered by each Standard of Part 1 of the ESG. When analys- 
ing the responses to these questions, it becomes clear that in the majority of heis, stu- 
dent representatives tend to be largely or even fully involved in the areas covered by 
Standards 1.1, 1.2, whilst being little or not involved in the areas covered by Standards 
1.3, 1.4, 1.6 and 1.7. With regard to the involvement in the areas covered by Standard 1.5 
Learning resources and student support, the responses are equally divided between 
those who indicate that they are asked to provide their feedback regarding the support 
services, and those who say they are not. 

In other words, although a direct correlation cannot be proven based on this data, the 
involvement of students in internal QA seems to be mostly limited to the areas where 
it is explicitly encouraged through the ESG. 

The most striking observation coming from the local student survey data on student 
involvement is however the large variety in which student involvement in QA is being 
implemented in different heis. Given the different cultures and history of student par- 
ticipation in the different countries the survey focussed on, diversity in the extent to 
which students are involved in QA was to be expected. However, there seems to be as 
much diversity within as between the countries. In all the case study countries, there 
are heis where students are largely involved in the different aspects of internal QA and 
those where they are (almost) not involved at all. 

The same pattern of almost complete diversity overall and between heis within the 
same country can be seen when it comes to the extent to which local student repre- 
sentatives are involved in the external review processes of heis and/or study pro- 
grammes. There seems to be some consistency within the different countries in the 
extent to which students are being consulted by an assessment panel during external 
reviews, but a large variation when it comes to their involvement in the overall ex- 
ternal review process, including the preparation of the self-evaluation report as well 
as the follow-up process cq. implementation of recommendations. When taking all 
the responses of the local student representatives together, there only appear to be as 
many heis who largely/fully involve students in the different steps of external review 
processes as those who (almost) do not involve students at all in these. 

Interestingly, about a third of the respondents do not know whether the external re- 
view panels also have a student member. Possible explanations for why they do not 
know this might again be a lack of involvement in the review process, but could also 
be because no reviews have taken place during their time as a student representative. 
Amongst the other respondents to the survey, slightly more than half answer that 
there is indeed an external student member in the review panels that undertake the 
external reviews of their university and/or study programmes. 
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In the reviews of QAAs, where students are, as a rule inscribed within the ESG, full 
members of the review panel, a majority of the students who responded to the survey 
felt they were an equal member of the team. However, some of them felt they were 
treated differently by the other team members and not always completely included 
fully in the whole process. 
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5 SUMMARY OF KEY FINDINGS 


Almost all the respondents to esus consultation declared that European cooperation 
in quality assurance is a necessity and they noted the ESG as the vital tool to base this 
cooperation on. The analysis of the data gathered shows that for students the general 
relevance of the ESG is undoubted. Respondents view the relevance of the ESG as high 
at the European level, national and institutional level alike. 

PURPOSE AND SCOPE 

The findings of the esu consultation process clearly indicate that the ESG fulfil a range 
of purposes, some explicitly mentioned in the introduction to the ESG as >esg pur- 
posesc, some not. Regardless of the variety of purposes of the ESG, there does not seem 
to be any contradiction between them. In fact, it seems to be one of the major success 
factors of the ESG that they are able to cater for different purposes. 

The ESG are regarded as highly relevant for the establishment of a sound QA system, 
both nationally and institutionally. Local and national respondents also attribute high 
relevance to the ESG providing criteria or a basis for them to be used for reviews of 
programmes, institutions and qaas, to making external quality assurance more trans- 
parent and simpler to understand and to contributing to a common frame of reference 
for the provision of he and QA in the ehea. 

When asked about what the main purpose of the ESG should be, students saw the ESG 
as an important tool in forming a consistent European Higher Education Area stand- 
ing on mutual trust and in ensuring the provision of incentives for changes that would 
lead to achievement of Bologna goals. Furthermore, the ESG should contribute to 
comparability and compatibility between higher education systems, yet they should 
not be too rigid in order to allow for adaptations to the national context. However, 
they are at the same time seen as a mechanism to maintain QA provisions irrespective 
of political changes in a country. Whilst the ESG are not a universal panacea and do 
not guarantee changes in national or institutional practices, the consulted students 
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believe that without the ESG, certain aspects would have only been incorporated sig- 
nificantly later or not at all into the higher education systems and heis — including 
practices such as new approaches to QA generally, the notion of quality culture, or 
student participation in QA. 

As mentioned above, the ESG are viewed as an important instrument for building the 
ehea and more concretely as an instrument that influences the development of insti- 
tutional quality culture and provides guidance for development of quality assurance 
procedures. Therefore, students see it as essential to widen the scope of the ESG in order 
to foster the development of the ehea in real terms and building trust between higher 
education systems and higher education institutions alike. Students see the necessity 
to widen the scope of the ESG in two ways. Firstly, within the current overall scope of 
the ESG the standards and guidelines should be amended to include clearer linkage 
and reference to the specific commitments undertaken within the Bologna Process 
since specific instruments outlined in the Bologna Process have a direct relevance for 
the quality of higher education which the ESG should help quality assurance to assure. 
Students explicitly stated that specific references to learning outcomes, qualification 
frameworks, student participation, ects and Diploma Supplement should be includ- 
ed in the relevant standards and guidelines. Furthermore, students assert that the 
overarching principles of the Bologna Process, particularly the value principles and 
commitments made to equal access and success as well as widening participation (so- 
cial dimension); move towards student centred learning; adoption of lifelong learning 
philosophy; and fostering of the mobility of students and staff should be referenced 
clearly in the ESG in order to support realisation of these aspects of qualitative higher 
education, while keeping in mind the limitations of direct impact the ESG themselves 
have as well. Secondly, students were of the opinion that further elements ought to be 
addressed by the ESG, namely institutional management, research, lifelong learning 
and transnational education. 

USABILITY AND CLARITY 

All the national student representatives have indicated that they are aware of the ESG. 
More surprisingly, perhaps, is that at the local level, only a few student representatives 
have indicated that they have not heard of the ESG. The majority of them indicate to 
have some knowledge about the ESG, even though not always in detail. The results of 
the analysis of the survey of national unions of students further show that a majority 
of the national unions find the ESG as a whole to be mostly understandable. However 
some parts have been identified as being difficult to understand. One of the issues that 
national unions identify as an obstacle to easy understanding and clarity of the ESG is 
the language, phrasing and in particular the meaning of the specific quality assurance 
terminology used in the ESG. This is even more the case for local unions of students, 
due to them being seldom exposed to discussions about QA at international level. 
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Students who have taken part in external reviews assert high familiarity with the ESG 
to the external review panel members. They also regard the language of the ESG as 
relatively easy to understand. Regardless, they highlight that the ESG leave ample 
room for ambiguity. About half of them indicated that there was little or no shared 
understanding between panel members in the external reviews of what the ESG mean, 
how they should be interpreted and how they should be used for reaching conclusions 
in reviews. Hence, findings and judgements of the external reviews may significantly 
depend on the composition of the teams or rather the individuals in the teams, which 
in turn may cause problems in terms of equity and reliability, as criteria are not inter- 
preted in a consistent manner when ESG are being used as a direct reference. 

Although for each of the standards in all three parts of the ESG national unions gave di- 
vergent views on their clarity, some standards can be highlighted as less understand- 
able than others. With regard to Part 1 of the ESG Standards 1.1 Policy and procedures 
for quality assurance and 1.4 Quality assurance of teaching staff were identified as 
those which were more difficult to understand compared to other standards. With 
regard to Part 2, this seems to be the case particularly with standard 2.2. Develop- 
ment of external quality assurance processes, 2.4 Processes fit for purpose, 2.5 Report- 
ing and 2.8 System-wide analysis. The individual students who had been involved in 
qaa reviews particularly pointed out that standard 2.1 was virtually never used in the 
external reviews in so far as Part 1 of the ESG has been looked at, indicating that the 
standard may not be clear or easyly understood in an unambiguous manner in the 
external reviews. This crucially indicates that the important link between internal and 
external quality assurance, i.e. parts 1, 2 and 3 of the ESG, is not implemented in a satis- 
fying way and it should thus be clarified how this link can practically be implemented 
and the importance of doing so should be made clear. 

As one of the reasons for the ambiguities identified in the ESG, the students in the 
focus groups and a careful text analysis of the ESG pointed clearly to the inconsistency 
of approach to individual standards and guidelines throughout the ESG. On one hand 
there are significant differences in the precision of formulation of different standards 
and their accompanying guidelines, while some are deemed rather precise and offer- 
ing little room for interpretation, others as vague and using terms that are much open 
to interpretation. Apart from the precision of formulations in individual standards 
and guidelines, a careful textual analysis of the ESG suggests that there are standards 
which may be vague or which are not elaborated much, while their accompanying 
guidelines are very concrete and bring in important, additional elements for imple- 
mentation and guidance. This is an important issue to address since students also not- 
ed that there is no clarity as to what the difference between standards and guidelines 
is supposed to be in practical use. In other words if guidelines do bring in a number of 
additional, important elements that are not addressed in the actual standard, to what 
extent should they be followed in the set-up of internal quality assurance or indeed 
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checked in external reviews. In addition, although practice gives an indication that 
standards are assigned different levels of priority, and are certainly prioritised over 
guidelines, it seems to be a matter of personal interpretation as to which standards 
are considered core and which less important. Thus it can be said the ESG as a docu- 
ment lack a coherent approach, resulting in inconsistencies in their application, which 
needs to be remedied. 

IMPLEMENTATION AND IMPACT 

The ESG have had considerable impact on the development of the national qa systems 
since their adoption, either directly or as a guiding document. Although in some cases, 
the ESG were used in detail when creating or changing the national quality assurance 
systems, the analysis of the data collected paints a very diverse picture of the way and 
extent to which standards and guidelines are actually being implemented within na- 
tional higher education systems and higher education institutions. 

The use that is sometimes made of the ESG in the development and changes of the 
quality assurance systems does not imply that the standards and guidelines have been 
implemented consistently. The analysis of the information received from the national 
unions suggests that all of the quality assurance systems have similar standards to 
those defined in the ESG as part of their systems, but at the same time they do lack 
some important elements of the standards or some standards as a whole in their sys- 
tems. 

A closer look at the institutional level shows that in all of the countries in our sam- 
ple, as diverse as they are, there are heis with a system of internal qa in line with the 
ESG, but more heis that have not. The responses from local student representatives 
suggests that the implementation of a qa system that follows the principles of the 
ESG relies more on heis than on national governments, and means that it is possible 
that where in one institution certain standards are almost completely translated into 
the internal QA system, another hei in the next town might have a system in which a 
number of standards and guidelines are lacking. 

The data gathered suggests that standards in Part 1 are more often identical or very 
similar to the standards defined in the national quality assurance systems for internal 
quality assurance within higher education institutions; this however does not mean 
that these standards are then also implemented within the individual heis. At the 
same time the national standards for the external quality assurance of higher educa- 
tion seem to be less often identical or similar to those defined in Part 2 of the ESG and 
most often differ in important elements or are not a part of the national quality assur- 
ance systems. 
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With regard to Part 1 of the ESG, standards 1.1 Policy and procedures for quality as- 
surance and 1.4 Quality assurance of teaching staff were identified as those where 
some important elements of these standards were not part of the national regulation 
or were not part of them at all. A considerable amount of national unions pointed out 
standards 1.6 Information systems and 1.7 Public information as being not fully imple- 
mented in the quality assurance systems in their country. Interestingly, the analysis of 
the data from institutional level provided by the local student union representatives 
indicated that standards 1.5 Learning resources and student support, 1.6 Information 
systems and 1.7 Public information are those that are in many cases not fully imple- 
mented in higher education institutions. A more nuanced picture exists of Standard 

1.4 qa of teaching staff, which has also often been identified as a standard which is 
not or only partly implemented in national regulation. With regards to this standard, 
students show a lack of awareness on one hand and scepticism about existence and 
functioning of staff procedures. 

With regard to Part 2, as mentioned earlier in Section 2, all of the standards are less 
often defined nationally in a similar manner to the ESG, but this seems to be the case 
particularly with standards 2.2. Development of external quality assurance processes, 

2.4 Processes fit for purpose, 2.5 Reporting and 2.8 System-wide analysis. 

With regard to Part 3, the standards 3.2 Official status, 3.4 Resources and 3.6 Inde- 
pendence seem to be most often undefined in national systems or having important 
elements missing. 

With regards to student involvement, the textual analysis of the ESG suggests that 
regardless if you refer to it as student involvement or student participation, or even 
student feedback, student engagement (yet another term often used) in qa can mean 
many different things to different people. 

Data obtained through the local student representative survey seems to support the 
impression that student participation is only relevant and encouraged in areas of qa 
where it is explicitly mentioned in the ESG. Although according to the respondents, 
in most heis there are clear regulations about student involvement in quality assur- 
ance, the number of respondents who indicate that the level of involvement is not suf- 
ficient is twice the number who are satisfied with it. Student representatives tend to 
be largely or even fully involved in the areas covered by Standards 1.1, 1.2, whilst being 
little or not involved in the areas covered by Standards 1.3, 1.4, 1.6 and 1.7. With regard 
to the involvement in the areas covered by Standard 1.5 Learning resources and stu- 
dent support, the responses are equally divided between those who indicate that they 
are asked to provide their feedback regarding the support services, and those who say 
they are not. 
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In other words, although a direct correlation cannot be proven based on this data, the 
involvement of students in internal qa seems to be mostly limited to the areas where it 
is explicitly encouraged through European Standards and Guidelines. 

The most striking observation coming from the local student survey data on student 
involvement is however the large variety in which student involvement in QA is being 
implemented in different heis. The same pattern of almost complete diversity overall 
and between heis within the same country can be seen when it comes to the extent 
to which local student representatives are involved in the external review processes 
of heis and/or study programmes. Given the different cultures and history of student 
participation in the different countries the survey focussed on, a diversity in the ex- 
tent to which student are involved in qa was to be expected. However, there seems to 
be as much diversity within as between the countries. In all the case study countries, 
there are heis where students are largely involved in the different aspects of internal 
and external qa and those where they are (almost) not involved at all. This indicates 
that in order to remedy the current situation of unequal possibilities for students to 
be involved in qa, a more specific mentioning of the principle of student participation 
in quality assurance should find its way into the Standards and Guidelines on internal 
and external qa themselves. 

CONCLUSIONS AND RECOMMENDATIONS 

The recommendations formulated in this chapter are based first and foremost on the 
views collected from all of the respondents on how to address the issues raised in the 
report, as well as on an overall data analysis and the authors’ views on and interpreta- 
tion of the key findings presented in this report. 

The consultation has shown that the ESG have had considerable impact on the devel- 
opment of the national QA systems since their adoption but that the way and extent 
to which standards and guidelines are actually being implemented within national 
higher education systems and higher education institutions varies significantly be- 
tween countries and individual heis and, overall, the standards and guidelines have 
not (yet) been implemented consistently. 

Most national quality assurance systems, where these have been developed thus far, 
include similar standards to those defined in the ESG, but at the same time they often 
lack some important elements of the standards or some standards. 

On the institutional level an even more diverse picture can be observed. There are 
heis with a system of internal QA in line with the ESG, but more heis without and the 
implementation of a qa system that follows the principles of the ESG relies more on 
heis than on national governments. 
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All of the respondents to this consultation agree that the ESG should be revised and 
amended and in the view of the authors, the issues that have emerged and their impli- 
cations form a clear basis for this and should therefore be addressed in the process of 
such a revision of the esg. 

It is essential that the revision of the ESG is not limited only to minor amendments 
of the existing standards and guidelines, but involves a broader discussion on the 
purposes and the scope of the ESG. This discussion on the purposes and the scope of 
the ESG may be more demanding and challenging than a mere improvement of the 
current text of the ESG — it is however essential. The standards and guidelines form 
a holistic document which is only meaningful in the context of its overall purpose. 
Furthermore, the time between the adoption of the ESG (May 2005) and the adoption 
of the revised ESG (2014) — if it were mandated by the Ministers responsible for he — 
implies that numerous developments in the ehea and the framework of the Bologna 
Process central to the quality of he are not addressed in the current ESG. Therefore, 
revising the ESG without revisiting the purposes and the scope of the ESG would con- 
stitute a missed opportunity for the E4 and the bfug to address many developments 
in the ehea in the past 6 or more years that are closely linked to the notion of quality 
of higher education. 

For students both at local and at national level not only the initially stated purposes 
of the ESG are important, but also other, de facto, purposes. This, in turn, poses the 
question whether the ESG are suitable to fulfil these purposes, given that they might 
not have been designed for them. This seems especially relevant regarding the use 
of the ESG as a checklist for reviews, since it has been stated at more than one occa- 
sion that the ESG are not intended for this. But in fact they are used in this manner, in 
particular at the level of reviews of Quality Assurance Agencies. It seems problematic 
that in practise an important purpose of the ESG is to provide criteria for reviews, but 
that they at the same time are not supposed to be seen as a compliance tool. It would 
be sensible to encourage a discussion about how to bridge this apparent discrepancy. 
Having in mind the role the ESG have played in the development of the national quali- 
ty assurance systems, also a discussion about the guiding factor of the ESG for national 
qa systems seems opportune. 

In addition, students expect the scope of the ESG to be widened in two ways. First- 
ly, within the current scope of the ESG, the commitments taken within the Bologna 
Process should be made more explicit. Specific instruments outlined in the Bologna 
Process should be linked to the ESG since these instruments have a direct relevance 
for the quality of higher education which the ESG should help quality assurance to ad- 
dress. These in particular include the commitment to learning outcomes as specified 
in the Berlin Communique, qualification frameworks, student participation, ects 
and the Diploma Supplement. Furthermore, students see the need for overarching 
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principles of the Bologna Process to be more clearly specified within the ESG. These 
would include the commitment to access to higher education, the social dimension of 
the Bologna Process, student-centred learning, lifelong learning and the mobility of 
students and staff. In terms of the ESG, this would mean that the standards and guide- 
lines should be revised either through amending the current standards and guidelines 
or developing new standards and guidelines that would address the abovementioned 
principles. 

Secondly, students have identified a number of issues of relevance that would widen 
the scope of the ESG as such. Students identified three main areas where they see a 
need to further elaborate the European cooperation in quality assurance through the 
ESG, namely in the areas of institutional management, research (including research- 
teaching links) and transnational education. 

Looking into the formulations of the ESG more concretely, the language that is used 
should be addressed. In order for the ESG to achieve their purposes, it is essential not 
only to create awareness and knowledge, but also for the ESG to be easily understand- 
able to higher education stakeholders active at the European, but crucially also at the 
national and institutional level. 

The clarity of language and terminology may be enhanced through elaborating further 
the differences between national contexts and approaches to quality assurance in the 
introduction to the ESG where more recent information and analysis of the national 
QA systems could be taken into account. The issue of ambiguity and achieving more 
easily a shared understanding of the ESG may be more complex to address. However, it 
is essential to address it, to enable consistent use of the ESG in external reviews of both 
higher education institutions and quality assurance agencies. Revising the ESG in a 
manner that will leave less room for interpretation might be a solution to this. But this 
would also imply less room for flexibility and enforce more rigidity. 

Although not a direct solution to the issue of clarity of the text of the ESG itself, further 
suggestions for improvement of the understandibility and usability of the ESG could 
be to include a glossary of terms in the context of the ESG purposes as well as a guide 
on how to use the ESG for external reviews and/or an explanatory report similar to that 
of the Lisbon Recognition Convention. 

To address the inconsistencies in the manner in which the standards and guidelines 
are written a revision of the standards and guidelines as such is needed. The revision 
of the actual individual standards and guidelines should at the very least address the 
inconsistencies in the level of precision in which different standards and accompa- 
nying guidelines are formulated. Moreover, the relationship between the standards 
and guidelines should be discussed, particularly having in mind that some guidelines 
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bring in new, additional elements to the standards. Thus, the role of the guidelines, 
particularly in the context of implementation and external reviews, should be clearly 
stated. There are different possible solutions to this, but the main effort should be to 
enforce an unambiguous and consistent interpretation and application of the ESG. 

With regards to student involvement, as mentioned in the key findings student in- 
volvement or student participation, or even student feedback or student engagement 
(yet another term often used) in qa can mean many different things to different peo- 
ple. The data collected through this study suggests that indeed student involvement 
is interpreted in different ways in different countries and even more surprisingly be- 
tween different higher education institutions within the same national system. 

Although according to the respondents, in most heis there are clear regulations about 
student involvement in quality assurance, the number of respondents who indicate 
that the level of involvement is not sufficient is twice the number who are satisfied with 
it. Furthermore, the local student representative survey seems to support the impres- 
sion that student participation is only relevant and to be encouraged in areas of QA 
where it is explicitly mentioned in the ESG. 

Therefore, in student views the student involvement should be defined more explicitly 
and consistently, both as a principle, and within different standards to fully integrate 
the principle of students as equal partners and active actors in higher education in- 
stitutions and higher education decision making processes, particularly those in qa. 
The Ministers summit in Bergen in 2005 where the Stocktaking report presented the 
student involvement in qa through specifying that it includes student participation in 
the governance of national bodies of qa, within teams for external review, consulta- 
tion or involvement during external reviews and involvement in internal evaluations 
already provides initial guidance for defining the student participation (Bologna 
Process Stocktaking, 2005, p. 17). 

Furthermore, for students it is essential that their role is acknowledged in the deci- 
sion making processes within higher education institutions and national education 
systems as equal partners in the development of quality higher education. This means 
that the students should be involved in the governance structures in higher education 
institutions and be consulted at the national level. In the context of this report, the 
students should be involved in the setting of both national and institutional quality 
assurance systems and take active part in the quality assurance bodies set either na- 
tionally or institutionally. In the view of students, the development of the ESG itself 
provides an excellent example of a stakeholder approach to a development of a Euro- 
pean instrument of cooperation in quality assurance in higher education. 
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